Geolocation notifications using augmented reality user devices

ABSTRACT

An augmented reality system that includes an augmented reality user device. The augmented reality user device includes a display for overlaying virtual objects onto objects in a real scene, a camera, and a global position system sensor. The augmented reality user device includes a processor implementing an object recognition engine, a virtual assessment engine, and a virtual overlay engine. The object recognition engine identifies a business from an image. The virtual assessment engine authenticates the user and identifies a user identifier for the user. The virtual assessment engine captures an image, performs object recognition on the image to identify the business, and generates a location identifier identifying the business. The virtual assessment engine sends a token with the user identifier and the location identifier to a remote server, receives geolocation notifications for the user from the server, and presents the geolocation notifications as virtual objects overlaid with a real scene.

TECHNICAL FIELD

The present disclosure relates generally to performing operations usingan augmented reality display device that overlays graphic objects withobjects in a real scene.

BACKGROUND

Throughout the course of a day, a person may need to aggregatinginformation from multiple sources to make decisions. Existingtwo-dimensional graphical user interfaces limit the amount ofinformation the person can see based on the size of the display. Inaddition, the person may have to interact with multiple windows orscreens on the graphical user interface in order to view all of theinformation the person is interested in. Using existing graphical userinterfaces and having to interact with multiple windows or screenscauses a disconnect between the information being present and a realworld environment.

Using existing systems, when a person is looking for information that islocated among different databases with different sources, the person hasto make data requests to each of the different sources in order toobtain the desired information. The process of making multiple datarequests to different data sources requires a significant amount ofprocessing resources to generate the data requests. Typically,processing resources are limited and the system is unable to performother tasks when processing resources are occupied which degrades theperformance of the system.

The process of sending multiple data requests and receiving informationfrom multiple sources occupies network resources until all of theinformation has been collected. This process poses a burden on thenetwork which degrades the performance of the network. Thus, it isdesirable to provide the ability to securely and efficiently aggregateinformation from multiple data sources.

SUMMARY

In one embodiment, the disclosure includes an augmented reality systemwith an augmented reality user device for a user. The augmented realityuser device has a display for overlaying virtual objects onto tangibleobjects in a real scene in real-time. The augmented reality user devicealso has a camera configured to capture images of tangible objects and aglobal position system (GPS) sensor configured to provide the geographiclocation of the user. The augmented reality user device further includesone or more processors connected to the display, the camera, and the GPSsensor.

The processors implement an object recognition engine, a virtualassessment engine, and a virtual overlay engine. The object recognitionengine is used to identify a business based on an image of the business.The virtual assessment engine authenticates a user based on a user inputand identifies a user identifier for the user in response toauthenticating the user. The virtual assessment engine generates alocation identifier identifying the business and generates a token thatincludes the user identifier, and the location identifier. The virtualassessment engine sends the token to a remote server and receivesvirtual assessment data with a geolocation notification identifying anoffer provided by the business in response to sending the token. Thevirtual overlay engine presents the geolocation notification as avirtual object overlaid within a real scene.

The augmented reality system further includes the remote server with ageolocation analysis engine. The geolocation analysis engine receivesthe token and determines account information for the user based on theuser identifier. The geolocation analysis engine identifies the offerprovided by the business based on the location identifier and theaccount information for the user and generates the geolocationnotification identifying the offer. The geolocation analysis enginegenerates the virtual assessment data that includes the geolocationnotification and sends the virtual assessment data to the augmentedreality user device.

In one embodiment, an augmented reality user device aggregatesinformation for a user looking at products in a store. The augmentedreality user device identifies the products the user is looking at andaggregates information related to a purchase of the products. Theaugmented reality user device presents information to the user asvirtual objects overlaid with the products in a real scene in front ofthe user. The aggregated information may include a price for theproducts, account information, offers (e.g. coupons or promotions),recommendations, and/or any other information for the user. Theaggregated information is customized for the user and based oninformation related to the user.

In another embodiment, an augmented reality user device aggregateinformation for a user looking at a product with a significant cost suchas a vehicle or an appliance. The augmented reality user deviceidentifies the product and aggregates information related to a purchaseof the product. The augmented reality user device presents informationto the user as virtual objects overlaid with the product in a real scenein front of the user. The aggregated information may include informationabout the product, information about other similar products, informationabout other vendors, payment options (e.g. existing accounts andavailable new accounts), and/or any other information for the user.

In yet another embodiment, an augmented reality user device aggregatesinformation for a user based on businesses located near the user, forexample, in front of the user. The augmented reality user deviceidentifies businesses the user is looking at and aggregates informationabout the business and/or products and services the business provides.The augmented reality user device identifies the businesses based ontheir geographic location and/or their features (e.g. signs andbranding). The aggregated information may include information about abusiness, information about offers on products or services provided bythe business, and/or any other information. The augmented reality userdevice presents information to the user as virtual objects overlaid withthe businesses in a real scene in front of the user.

The present embodiment presents several technical advantages. In oneembodiment, an augmented reality user device allows a user to reduce thenumber of requests used to obtain information from multiple datasources. Additionally, the augmented reality user device allows the userto authenticate themselves which allows the user to request and obtaininformation that is specific to the user and without having to providedifferent credentials to authenticate the user with each data source.

The amount of processing resources used for the reduced number of datarequests is significantly less than the amount of processing resourcesused by existing systems. The overall performance of the system isimproved as a result of consuming less processing resources. Reducingthe number of data requests also reduces the amount of data traffic usedto obtain information from multiple sources which results in improvednetwork utilization and network performance.

The augmented reality user device generates tokens based on the identityof a user and the location of the user which improves the performance ofthe augmented reality user device by reducing the amount of informationused to request. Tokens are encoded or encrypted to obfuscate and maskinformation being communicated across a network. Masking the informationbeing communicated protects users and their information in the event ofunauthorized access to the network and/or data occurs.

The augmented reality user device uses object recognition and opticalcharacter recognition to identify the location of the user and/orobjects the user is looking at. Retrieving information about thelocation of the user and objects the user is looking at using objectrecognition and optical character recognition allows the augmentedreality user device to reduce the amount of time involved to make a datarequest compared to existing systems that rely on the user to manuallyenter all of the information for a request. This process for collectinginformation for the data request also reduces the likelihood of userinput errors and improves the reliability of the system.

Another technical advantage is the augmented reality user device allowsa user to view information as a virtual or graphic object overlaid ontotangible objects in the real scene in front of the user. This allows theuser to quickly view information in the context of the actual scene infront of the user.

Certain embodiments of the present disclosure may include some, all, ornone of these advantages. These advantages and other features will bemore clearly understood from the following detailed description taken inconjunction with the accompanying drawings and claims.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of this disclosure, reference is nowmade to the following brief description, taken in connection with theaccompanying drawings and detailed description, wherein like referencenumerals represent like parts.

FIG. 1 is a schematic diagram of an embodiment of an augmented realitysystem configured to overlay virtual objects with a real scene;

FIG. 2 is a first person view of an embodiment for an augmented realityuser device overlaying virtual objects with a real scene;

FIG. 3 is a first person view of another embodiment for an augmentedreality user device overlaying virtual objects with a real scene;

FIG. 4 is a schematic diagram of an embodiment of an augmented realityuser device employed by the augmented reality system;

FIG. 5 is a flowchart of an embodiment of an augmented realityoverlaying method;

FIG. 6 is a flowchart of another embodiment of an augmented realityoverlaying method;

FIG. 7 is a flowchart of another embodiment of an augmented realityoverlaying method;

FIG. 8 is a flowchart of another embodiment of an augmented realityoverlaying method;

FIG. 9 is a schematic diagram of another embodiment of an augmentedreality system configured to overlay virtual objects with a real scene;

FIG. 10 is a flowchart of another embodiment of an augmented realityoverlaying method; and

FIG. 11 is a flowchart of another embodiment of an augmented realityoverlaying method.

DETAILED DESCRIPTION

Throughout the course of a day a person may need to aggregate differentkinds of information from multiple sources, for example, to make variousdecision. For example, the person may want to look-up their personalinformation, information about the location of the person, informationabout an object, or any other information. All of this information maybe located in different databases with different sources which resultsin several technical problems.

Using existing systems, the person has to make individual data requeststo each of the different sources in order to obtain the desiredinformation. This process involves making numerous data requests todifferent data sources which uses a significant amount of processingresources to generate the data requests. Typically processing resourcesare limited and the system is unable to perform other tasks whenprocessing resources are occupied which degrades the performance of thesystem. The process of sending numerous data requests and receivinginformation from multiple sources occupies network resources until allof the information has been collected. This process poses a burden onthe network which degrades the performance of the network.

Additionally, each data request may use different credentials toauthenticate the person with each of the different sources. Providingdifferent credentials to each source increases the complexity of thesystem and increases the amount of data that is sent across the network.The increased complexity of the system makes existing systems difficultto manage. The additional data that is sent across the network bothoccupies additional network resources and exposes additional sensitiveinformation to network.

A technical solution to these technical problems is an augmented realityuser device that allows a user to reduce the number of data requestsused to obtain information from multiple sources. The augmented realityuser device allows the user to process an image to extract informationfrom for the data request. The augmented reality user device allows theuser to authenticate themselves to obtain information that allows theuser to request and obtain personal information that is specific to theuser with the same data request. The amount of processing resources usedto generate the reduced number of data requests is significantly lessthan the amount of processing resources used by existing systems togenerate numerous data requests. The overall performance of the systemis improved as a result of consuming less processing resources. Using areduced number of data requests to obtain information from multiplesources reduces the amount of data traffic used to obtain theinformation which results in improved network utilization and networkperformance.

Securely transferring data and information across a network posesseveral technical challenges. Networks are susceptible to attacks byunauthorized users trying to gain access to sensitive information beingcommunicated across the network. Unauthorized access to a network maycompromise the security of the data and information being communicatedacross the network.

One technical solution for improving network security is an augmentedreality user device that generates and uses tokens that are used byaugmented reality user device to request potentially sensitiveinformation. The augmented reality user device allows tokens to begenerated automatically upon identifying and extracting information froman image. The token may be encoded or encrypted to obfuscate theinformation being communicated by it. Using tokens to mask informationthat is communicated across the network protects users and theirinformation in the event of unauthorized access to the network and/ordata occurs. The tokens also allow for data transfers to be executedusing less information than other existing systems, and thereby reducesthe amount of data that is communicated across the network. Reducing theamount of data that is communicated across the network improves theperformance of the network by reducing the amount of time networkresource are occupied.

The augmented reality user device uses object recognition and opticalcharacter recognition of images to quickly retrieve information forgenerating tokens. The augmented reality user device allows informationfor generating tokens to be retrieved based on an image of an objectwhich significantly reduces the amount of time involved to make a datarequest compared to existing systems that rely on the user to manuallyenter all of the information for the request. Using object recognitionand optical character recognition to identify and to retrieveinformation also allows the augmented reality user device to be lessdependent on user input, which reduces the likelihood of user inputerrors and improves reliability of the system.

Another technical challenge of using existing systems is the usage oftwo-dimensional graphical user interfaces. Existing two-dimensionalgraphical user interfaces limit the amount of information the person cansee based on the size of the display. In addition, the person may haveto interact with multiple windows or screens on the graphical userinterface in order to view all of the information the person isinterested in. Using existing graphical user interfaces and having tointeract with multiple windows or screens causes a disconnect betweenthe information being present and a real world environment.

An augmented reality user device allows a user view information asvirtual or graphical object overlaid onto the physical object inreal-time. For example, using the augmented reality user device, theuser is able to quickly view information about one or more objects thatare in front of the user. The user is able to view information about theobjects, their personal information, and/or the location of the user asa virtual object overlaid onto any tangible objects in the real scene infront of the user.

FIGS. 1 and 9 illustrate different examples of a user employing anaugmented reality user device to view virtual objects overlaid withtangible objects in a real scene in front of the user. FIG. 1illustrates an example of an augmented reality system configured toaggregate information about one or more products. FIG. 9 illustrates anexample of an augmented reality system configured to aggregateinformation about a business.

FIGS. 2 and 3 provide examples of a first person view of what a usermight see when using the augmented reality user device to view virtualobjects overlaid with tangible objects. FIG. 4 is an embodiment of howan augmented reality user device may be configured and implemented.FIGS. 5, 7, and 10 are examples of a process for facilitating augmentedreality overlays with tangible objects using an augmented reality userdevice. FIGS. 6, 8, and 11 are examples of a process for facilitatingaugmented reality overlays with tangible objects with a remote server.

FIG. 1 is a schematic diagram of an embodiment of an augmented realitysystem 100 configured to overlay virtual objects with a real scene. Theaugmented reality system 100 comprises an augmented reality user device400 in signal communication with a remote server 102 and one or morethird-party databases 118 via a network 104. The augmented reality userdevice 400 is configured to employ any suitable connection tocommunicate data with the remote server 102. In FIG. 1, the augmentedreality user device 400 is configured as a head-mounted wearable device.Other examples of wearable devices are integrated into a contact lensstructure, an eye glass structure, a visor structure, a helmetstructure, or any other suitable structure. In some embodiments, theaugmented reality user device 400 may be integrated with a mobile userdevice. Examples of mobile user devices include, but are not limited to,a mobile phone, a computer, a tablet computer, and a laptop computer.For example, the user 106 may use a smart phone as the augmented realityuser device 400 to overlay virtual objects with a real scene. Additionaldetails about the augmented reality user device 400 are described inFIG. 4.

Examples of an augmented reality user device 400 in operation aredescribed below and in FIGS. 5, 7, and 10. The augmented reality userdevice 400 is configured to identify and authenticate a user 106 and toprovide a user identifier 108 that identifies the user 106. The useridentifier 108 is a label or descriptor (e.g. a name based onalphanumeric characters) used to identify the user 106. The augmentedreality user device 400 is configured to use one or more mechanisms suchas credentials (e.g. a log-in and password) or biometric signals toidentify and authenticate the user 106.

The augmented reality user device 400 is configured to identify thelocation of the user 106. In one embodiment, the augmented reality userdevice 400 identifies the location of the user 106 based on thegeographic location of the user 106. For example, the augmented realityuser device 400 uses geographic location information provided by aglobal position system (GPS) sensor with a map database to determine thelocation of the user 106. In another embodiment, the augmented realityuser device 400 is configured to use object recognition and/or opticalcharacter recognition to identify the location of the user 106. Forexample, the augmented reality user device 400 is configured to identifythe location of the user 106 based on the identification of buildings,structures, landmarks, branding, signs, logo, and/or any other types ofobjects around the user. In other embodiments, the augmented realityuser device 400 determines the location of the user 106 based on anyother information and/or using any other suitable technique as would beappreciated by one of ordinary skill in the art.

The augmented reality user device 400 is configured to identify tangibleobjects 150 in front of the user 106. For example, the augmented realityuser device 400 is configured to identify products in a shopping cart152. The augmented reality user device 400 is configured to use objectrecognition and/or optical character recognition to identify objects150. In one embodiment, the augmented reality user device 400 isconfigured to capture an image 407 of an object 150 and to performobject recognition and/or optical character recognition on the image 407of the object 150 to identify the object 150. The augmented reality userdevice 400 is configured to identify an object 150 based on the size,shape, features, and/or any other characteristics of the object 150. Forexample, the augmented reality user device 400 identifies the object 150based on branding, text, or logos on the object 150 or its packaging.The augmented reality user device 400 identifies the object 150 based onany other characteristics of the object 150 or using any other suitabletechnique as would be appreciated by one of ordinary skill in the art.The augmented reality user device 400 uses object recognition and/oroptical character recognition which allows the user 106 to quickly andefficiently identify the objects 150 compared to other systems where theuser 106 has to manually identify each of the objects 150.

The augmented reality user device 400 is further configured to determinea price for an object 150 or a price for one or more objects 150. In oneembodiment, the augmented reality user device 400 accesses a third-partydatabase 118 to determine the price for each of the objects 150. Forexample, the augmented reality user device 400 queries a third-partydatabase 118 linked with a vendor of the one or more objects 150 todetermine the price of the one or more objects 150. In one embodiment,the augmented reality user device 400 sends a message 113 identifyingthe one or more objects 150 to the third-party database 118. Forexample, the message 113 comprises descriptors for the objects 150.Examples of descriptors include, but are not limited to, images 407 ofthe objects 150, names, barcodes, object descriptors (e.g. type, size,or weight), and/or any other suitable descriptor for identifying theobjects 150.

In another embodiment, the augmented reality user device 400 determinesthe price of the objects 150 using barcodes, quick response (QR) codes,digital watermarks, or price tags associated with the objects 150. Forexample, the augmented reality user device 400 identifies price tags onthe objects 150 and determines the price of the objects 150 based ontheir price tags. As another example, the augmented reality user device400 identifies price tags on the shelves when the user 106 picks up anobject 150. In another embodiment, the augmented reality user device 400determines the total price of the objects 150 based on an input providedby the user 106. The augmented reality user device 400 is configured toreceive an indication of prices for the objects 150 from the user 106 asa voice command, a gesture, an interaction with a button on theaugmented reality user device 400, or in any other suitable form. Forexample, the augmented reality user device 400 receives voice commandsfrom the user 106 as the user 106 states the prices of each of theobjects 150. In other examples, the augmented reality user device 400determines the price of the objects 150 using any other suitabletechnique as would be appreciated by one of ordinary skill in the art.

The augmented reality user device 400 is configured to generate a token110 for requesting information for the user 106. In one embodiment, thetoken 110 comprises a user identifier 108 for the user 106, one or moreproduct identifiers 115, a vendor identifier 117, and a price 109corresponding with the product identifiers 115. The one or more productidentifiers 115 identify one or more objects 150. The vendor identifier117 identifies where the user 106 is located. For example, the productidentifiers 115 identify several groceries and a vendor identifier 117identifies a grocery store. As another example, the product identifiers115 identify office supplies and a vendor identifier 117 identifies anoffice supply store. In other embodiments, the token 110 identifies anyother information or combination of information.

The augmented reality user device 400 is configured to send the token110 to the remote server 102. In one embodiment, the augmented realityuser device 400 is configured to encrypt and/or encode the token 110prior to sending the token 110 to the remote server 102. The augmentedreality user device 400 employs any suitable encryption and/or encodingtechnique as would be appreciated by one of ordinary skill in the art.

The augmented reality user device 400 is further configured to receivevirtual assessment data 111 from the remote server 102 in response tosending the token 110 to the remote server 102. The augmented realityuser device 400 is configured to process the virtual assessment data 111to access the information provided by the remote server 102. The virtualassessment data 111 comprises information related to account informationfor the user 106, recommendations for the user 106, available offers forthe user 106, and/or any other information for the user 106.

The augmented reality user device 400 is configured to presentinformation from the received virtual assessment data 111 as one or morevirtual objects overlaid with the tangible objects 105 in the real scenein front of the user 106. For example, the augmented reality user device400 presents account information for the user 106 and a recommendationidentifying one of the user's 106 accounts as virtual objects overlaidwith the objects 150 in front of the user 106. Examples of the augmentedreality user device 400 presenting information as virtual objectsoverlaid with the objects 150 in front of the user 106 are described inFIGS. 2 and 3. Using virtual objects overlaid, the augmented realityuser device 400 provides a head-up display that allows the user 106 toview their personal information while simultaneously looking at otherobjects in front of the user 106.

The augmented reality user device 400 is configured to determine whetherthere are any offers available for the user 106. In one embodiment, theaugmented reality user device 400 determines there are offers availablefor the user 106 based on the presence of the offers in the receivedvirtual assessment data 111. The augmented reality user device 400 isconfigured to present available offers for the user 106 as virtualobjects overlaid with the objects 150 in front of the user 106. Forexample, the augmented reality user device 400 overlays a virtual objectidentify an offer for a product with the actual product in the realscene. The augmented reality user device 400 allows the user 106 toquickly identify offers in the context of the real scene. Other systemsare unable to link offers with actual products which forces a shopper tomanually determine which offers apply to which products.

In one embodiment, one or more of the available offers involvesactivation by the user 106 in order to be used by the user 106. Theaugmented reality user device 400 is further configured to determinewhether the user 106 selects an offer to activate. The user 106 selectsor identifies an offer from among the one or more available offers whenthe user 106 wants to activate the offer. The augmented reality userdevice 400 is configured to receive an indication of the selected offerfrom the user 106 as a voice command, a gesture, an interaction with abutton on the augmented reality user device 400, or in any othersuitable form. The augmented reality user device 400 is configured tosend an activation command 128 identifying the selected offer to theremote server 102 to activate the offer. The augmented reality userdevice 400 allows the both see available offers and accept one or moreoffers while simultaneously looking at other objects in front of theuser 106. This allows the user 106 to continue other activities withoutinterruption.

The network 104 comprises a plurality of network nodes configured tocommunicate data between the augmented reality user device 400 and oneor more servers 102 and/or third-party databases 118. Examples ofnetwork nodes include, but are not limited to, routers, switches,modems, web clients, and web servers. The network 104 is configured tocommunicate data (e.g. tokens 110 and virtual assessment data 111)between the augmented reality user device 400 and the server 102.Network 104 is any suitable type of wireless and/or wired networkincluding, but not limited to, all or a portion of the Internet, thepublic switched telephone network, a cellular network, and a satellitenetwork. The network 104 is configured to support any suitablecommunication protocols as would be appreciated by one of ordinary skillin the art upon viewing this disclosure.

The server 102 is linked to or associated with one or more institutions.Examples of institutions include, but are not limited to, organizations,businesses, government agencies, financial institutions, anduniversities, among other examples. The server 102 is a network devicecomprising one or more processors 112 operably coupled to a memory 114.The one or more processors 112 are implemented as one or more centralprocessing unit (CPU) chips, logic units, cores (e.g. a multi-coreprocessor), field-programmable gate array (FPGAs), application specificintegrated circuits (ASICs), or digital signal processors (DSPs). Theone or more processors 112 are communicatively coupled to and in signalcommunication with the memory 114. The one or more processors 112 areconfigured to process data and may be implemented in hardware orsoftware. The one or more processors 112 are configured to implementvarious instructions. For example, the one or more processors 112 areconfigured to implement a product analysis engine 116. In an embodiment,the product analysis engine 116 is implemented using logic units, FPGAs,ASICs, DSPs, or any other suitable hardware.

Examples of the product analysis engine 116 in operation are describedin detail below and in FIGS. 6 and 8. In one embodiment, the productanalysis engine 116 is configured to receive a token 110 and to processthe token 110 to identify a user identifier 108 for the user 106, one ormore product identifiers 115 identifying one or more objects 150, avendor identifier 117 identifying a vendor, a price 109 for the one ormore objects 150, and/or any other information. In one embodiment,processing the token 110 comprises decrypting and/or decoding the token110 when the token 110 is encrypted or encoded by the augmented realityuser device 400. The product analysis engine 116 employs any suitabledecryption or decoding technique as would be appreciated by one ofordinary skill in the art.

The product analysis engine 116 is configured to use the user identifier108 to look-up and identify account information for the user 106 in anaccount information database 120. The account information comprises oneor more accounts (e.g. payment accounts), budgeting information,transaction history, membership information (e.g. loyalty or rewardprogram memberships), and/or any other information linked with the user106. Examples of accounts include, but are not limited to, checkingaccounts, savings accounts, investment accounts, credit card accounts,lines of credit, and any other suitable type of account. Budgetinginformation comprises one or more budgets with recommended oruser-defined spending limits for the user 106. For example, thebudgeting information comprises a grocery budget, a clothing budget, arestaurant budget, and/or any other kind of budget.

The product analysis engine 116 is configured to determine whether thereare any available offers for the user 106 based on information providedby the token 110 and/or the account information for the user 106.Examples of offers include, but are not limited to, promotions, coupons,in-store credit, vouchers, rebates, membership points, reward points,air travel miles, free product vouchers, and/or any other kind of offer.In one embodiment, the product analysis engine 116 is configured to usethe user identifier 108 and/or the account information for the user 106to query an offers database 122 for any available offers for the user106. For example, the product analysis engine 116 uses the useridentifier 108 to look-up and identify offers that are available for theuser 106.

In another embodiment, the product analysis engine 116 is configured tosend a data request 126 comprising information provided by the token 110and/or account information for the user 106 to one or more third-partydatabases 118 to query the third-party databases 118 for availableoffers for the user 106. For example, a third-party database 118 islinked with the vendor identified by the vendor identifier 117 andprovides available offers from the vendor in response to the datarequest 126. In one embodiment, the data request 126 comprises the useridentifier 108, account information for the user 106, informationprovided by the token 110, any other information linked with the user106, or combinations of information.

The product analysis engine 116 is configured to generate personalizedrecommendations for the user 106 based on the information provided bythe token 110 and account information for the user 106. For example, theproduct analysis engine 116 generates recommendations that save the user106 money, earns the most rewards or loyalty points, keeps the user 106on budget, prevents the user 106 from over-drafting, and/or provides anyother kinds of benefits for the user 106. As an example, arecommendation identifies one of the user's 106 accounts for atransaction. In this example, the product analysis engine 116 generatesthe recommendation that prevents the user 106 from over-drafting orincurring fees. As another example, a recommendation identifies one ormore offers available to the user 106 for a transaction. In thisexample, the product analysis engine 116 generates the recommendationthat earns the most reward points or provides a discount for the user106. As another example, a recommendation indicates for the user 106 toremove one or more objects 150 to avoid going over budget orover-drafting. In one embodiment, the product analysis engine 116generates a recommendation for the user 106 based on the user's 106account information and/or the available offers for the user 106. Forexample, the product analysis engine 116 generates a recommendationbased on account balances, budget information for the user 106, rewardpoints that will be earned, coupons that can be applied, and/or anyother criteria.

The product analysis engine 116 is configured to generate virtualassessment data 111 that comprises a recommendation, account informationfor the user 106, one or more available offers for the user 106,alternative vendors with alternative prices, historical pricinginformation for a product, and/or any other information for the user106. The virtual assessment data 111 comprises information to bepresented to the user 106 as one or more virtual objects. The productanalysis engine 116 sends the virtual assessment data 111 to theaugmented reality user device 400.

The product analysis engine 116 is further configured to receive anactivation command 128 identifying a selected offer by the user 106. Theproduct analysis engine 116 is configured to identify the selected offerand to facilitate activating the selected offer for the user 106. Forexample, the product analysis engine 116 is configured to exchangemessages with a third-party database 118 to activate the selected offerfor the user 106. Once an offer is activated, the user 106 may redeem oruse the selected offer. In one embodiment, the product analysis engine116 is configured to send virtual assessment data 111 to the augmentedreality user device 400 that indicates the selected offer has beenactivated.

The memory 114 comprises one or more disks, tape drives, or solid-statedrives, and may be used as an over-flow data storage device, to storeprograms when such programs are selected for execution, and to storeinstructions and data that are read during program execution. The memory114 may be volatile or non-volatile and may comprise read-only memory(ROM), random-access memory (RAM), ternary content-addressable memory(TCAM), dynamic random-access memory (DRAM), and static random-accessmemory (SRAM). The memory 114 is operable to store an accountinformation database 120, offers database 122, product informationdatabase 123, product analysis instructions 124, and/or any other dataor instructions. The product analysis instructions 124 comprise anysuitable set of instructions, logic, rules, or code operable to executethe product analysis engine 116.

The account information database 120 comprises account information forthe user 106. Account information includes, but is not limited to,personal information, credit scores, credit history, institution names,account names, account balances, account types, budget information,rewards points, member benefits, transaction history, and paymenthistory.

The offers database 122 is configured to store offers for the user 106,vendors, and/or one or more products. In one embodiment, the offersdatabase 122 is configured to receive offers from one or morethird-party databases 118. In other embodiments, the offers database 122is configured to generate offers for the user 106. Examples of offersinclude, but are not limited to, promotions, coupons, in-store credit,vouchers, rebates, membership points, reward points, air travel miles,free product vouchers, and/or any other kind of offer. In someembodiments, an offer involves activation by the user 106 prior to usingthe offer. For example, the offer becomes valid in response to the user106 accepting the offer. Offers are linked with and can be identifiedusing account information for the user 106, user identifiers 108,product identifiers 115, and/or vendor identifiers 117. For example, theoffers database 122 uses a user identifier 108 to look-up and identifyoffers available for the user 106. In an other example, the offersdatabase 122 uses product identifiers 115 to look-up and identify offerslinked with one or more products. In another example, the offersdatabase 122 uses a vendor identifier 117 to look-up and identify offerslinked with a particular vendor. In other examples, the offers database122 uses any other information to look-up and identify offers for theuser 106.

The product information database 123 is configured to store productinformation for various objects 150. In one embodiment, the productinformation comprises product reviews, product descriptions, warrantyinformation, historical pricing information, seasonal pricinginformation, secondary costs (e.g. insurance costs and maintenancecosts), demographic information about other people who purchased asimilar product, information about other models of a product,alternative products, alternative vendors with alternative prices for aproduct, any other information, or combination of information. In oneembodiment, the product analysis engine 116 is configured to use productidentifiers 115 to look-up and identify product information. Forexample, the product analysis engine 116 uses a product identifier 115to identify alternative vendors with alternative pricing for a productlinked with the product identifier 115. In some embodiments, the productanalysis engine 116 is configured to send a data request 126 comprisinga product identifier 115 to a third-party database 118 to requestproduct information.

In an embodiment, the account information database 120, the offersdatabase 122, and/or the product information database 123 are stored ina memory external of the server 102. For example, the server 102 isoperably coupled to a remote database storing the account informationdatabase 120, the offers database 122, and/or the product informationdatabase 123.

In one embodiment, the server 102 is in signal communication with one ormore third-party databases 118. Third-party databases 118 are databasesowned or managed by a third-party source. Examples of third-partysources include, but are not limited to, vendors, institutions, loyaltyprograms, and businesses. In one embodiment, the third-party databases118 are configured to store account information for the user 106, offersfor the user 106, offers for one or more products, offers for a vendor,mapping information, product information, vendor information, taxinformation, and/or any other information. In one embodiment,third-party databases 118 are configured to push (i.e. send) data to theserver 102. The third-party database 118 is configured to sendinformation (e.g. offers) to the server 102 with or without receiving adata request for the information. The third-party database 118 isconfigured to send data periodically to the server 102, for example,hourly, daily, or weekly. For example, the third-party database 118 isassociated with a vendor and is configured to push offers linked thevendor and/or products sold by the vendor to the server 102 hourly.

In another embodiment, a third-party database 118 is configured toreceive a data request 126 for information or offers linked with avendor, one or more products, and/or the user 106 from the server 102.The third-party database 118 is configured to send the requestedinformation back to the server 102. For example, a third-party database118 is configured to receive a data request 126 comprising a useridentifier 108 for the user 106. The third-party database 118 isconfigured to use the user identifier 108 to look-up offers availablefor the user 106 within the records of the third-party database 118. Inanother example, a third-party database 118 is configured to receive adata request 126 comprising a product identifier 115. The third-partydatabase 118 is configured to use the product identifier 115 linked witha product to look-up offers for the product within the records of thethird-party database 118. In other examples, third-party databases 118are configured to use any information provided to the server 102 tolook-up offers linked with a vendor, one or more products, and/or theuser 106.

In one embodiment, the third-party databases 118 are configured toreceive a message 113 comprising descriptors for one or more objects 150from the augmented reality user device 400. The third-party databases118 are configured to use the descriptors to look-up prices for the oneor more objects 150 linked with the descriptors. The third-partydatabases 118 are configured to send prices for the one or more objects150 to the augmented reality user device 400.

The following is a non-limiting example of how the augmented realitysystem 100 may operate when a user 106 is looking at several products ina store. In this example, a user 106 is using the augmented reality userdevice 400 in a store (e.g. a grocery store). The user 106 authenticatesthemselves before using the augmented reality user device 400 byproviding credentials (e.g. a log-in and password) and/or a biometricsignal. The augmented reality user device 400 authenticates the user 106based on the user's input and allows the user 106 to generate and sendtokens 110. The augmented reality user device 400 identifies the user106 and a user identifier 108 for the user 106 upon authenticating theuser 106. Once the user 106 has been authenticated, the user identifier108 is used by other systems and devices (e.g. remote server 102 or athird-party database 118) to identify and authenticate the user 106without requiring the user 106 to provide additional credentials foreach system.

Once the user 106 is authenticated, the augmented reality user device400 identifies the location of the user 106, for example, the storewhere the user 106 is shopping. In one embodiment, the augmented realityuser device 400 identifies the location of the user 106 based on thegeographic location of the user 106. For example, the augmented realityuser device 400 uses geographic location information provided by a GPSsensor with a map database (e.g. a third-party database 118) todetermine the location of the user 106 and to identify the store wherethe user 106 is shopping. In another embodiment, the augmented realityuser device 400 uses object recognition and/or optical characterrecognition to identify the store. For example, the augmented realityuser device 400 identifies the store based on structures, branding,signs, and/or logos within the store. In other embodiments, theaugmented reality user device 400 identifies the store using any othersuitable technique.

The user 106 looks at objects 150 (e.g. products) in a shopping cart 152with the augmented reality user device 400. The augmented reality userdevice 400 captures an image 407 of the objects 150 in the shopping cart152 and performs object recognition and/or optical character recognitionon the captured image 407 to identify the objects 150 in the shoppingcart 152. For example, the augmented reality user device 400 identifiesthe objects 150 based on the size and shape of the objects 150 or basedon text or logos on their packaging.

The augmented reality user device 400 determines the price of theobjects 150 in the shopping cart 152. In one embodiment, the augmentedreality user device 400 accesses a third-party database 118 linked withthe store to determine the price of the objects 150. For example, theaugmented reality user device 400 queries the third-party database 118using descriptors for the identified objects 150 in the shopping cart152. In another embodiment, the augmented reality user device 400determines a price of the objects 150 by identifying barcodes, QR codes,digital watermarks, or price tags associated with the objects 150. Forexample, the augmented reality user device 400 identifies price tags onthe objects 150. As another example, the augmented reality user device400 identifies a price tag on a shelf for an object 150 when the user106 picks up the object 150.

The augmented reality user device 400 generates a token 110 and sendsthe token 110 to the remote server 102. In one embodiment, the augmentedreality user device 400 generates a token 110 comprising the useridentifier 108 for the user 106, a vendor identifier 117 identifying thestore where the user 106 is located, product identifiers 115 identifyingthe objects 150 in the shopping cart 152, and the price of the objects150. In other embodiments, the token 110 comprises any other suitableinformation or combinations of information. The augmented reality userdevice 400 encrypts and/or encodes the token 110 prior to sending thetoken 110 to the remote server 102.

The server 102 receives the token 110 and processes the token 110 toidentify the user identifier 108, the vendor identifier 117, the productidentifiers 115, and the price of the objects 150. The server 102decrypts or decodes the token 110 when the token 110 is encrypted orencoded by the augmented reality user device 400. The server 102 usesthe user identifier 108 to look-up account information and/or accountsfor the user 106 in the account information database 120. The server 102is configured to use the user identifier 108 to identify one or moreaccounts for the user 106.

In one embodiment, the server 102 uses the user identifier 108 toidentify budget information for the user 106 in the account informationdatabase 120. The budget information comprises one or more budgetsidentifying recommended or user-defined spending limits for the user106. For example, the server 102 identifies one or more stored budgetsfor the user 106, for example, a grocery budget, a clothing budget, arestaurant budget, and/or any other kinds of budgets. In one embodiment,the server 102 determines how the user's 106 accounts and budgets areimpacted by purchasing the products linked with the product identifiers115 based on the price 109 indicated in the token 110.

The server 102 determines whether there are any offers available for theuser 106 based on the user's 106 account information, the location ofthe user 106 (e.g. the vendor identifier 117), and/or the products inthe user's 106 shopping cart 152. In one embodiment, the server 102queries the offers database 122 for any available offers based on theuser's 106 account information. For instance, the server 102 queries theoffers database 122 to determine whether the user 106 has reward pointsor discounts that can be applied to one or more of the products in theuser's 106 shopping cart 152. In another embodiment, the server 102sends a data request 126 to one or more third-party databases 118 toquery the one or more third-party databases 118 for available offersbased on the user's 106 identity (e.g. the user identifier 108), theproducts in the user's 106 shopping cart 152 (e.g. the productidentifiers 115), and/or the store where the user 106 is shopping (e.g.the vendor identifier 117). For instance, a third-party database 118linked with the store identified by the vendor identifier 117 andprovides coupons or discounts for one or more of the products in theuser's shopping cart 152.

The server 102 generates a personalized recommendation for the user 106based on the user's 106 account information and/or available offers forthe user 106. As an example, the server 102 generates a recommendationthat identifies one of the user's 106 accounts for a transaction topurchase the products in the user's 106 shopping cart 152. As anotherexample, the server 102 generates a recommendation that identifies oneor more offers (e.g. coupons) that can be automatically applied. Asanother example, the server 102 generates a recommendation thatidentifies one or more offers that can be applied after activation. Inanother embodiment, the recommendation may indicate for the user 106 toremove one or more products from their shopping cart 152. For example,the server 102 may determine based on the price 109 indicated in thetoken 110 that purchasing the current products 150 will exceed theuser's 106 budget or overdraft the user's 106 account. The server 102 isconfigured to make other suitable kind of recommendation.

The server 102 generates virtual assessment data 111 that comprises therecommendation, account information for the user 106, one or moreavailable offers for the user 106, and/or any other information. Forexample, the server 102 generates virtual assessment data 111 thatcomprises the account information and one or more available offers forthe user 106. In this example, the virtual assessment data 111identifies different accounts available for the user 106 and one or moreavailable offers the user 106 may be interested in. As another example,the server 102 generates virtual assessment data 111 that comprisesaccount information and budgeting information for the user 106. In thisexample, the virtual assessment data 111 identifies different accountsavailable for the user 106 and how the user's 106 budget is affected bypurchasing the products in the user's shopping cart 152. As anotherexample, the server 102 generates virtual assessment data 111 thatcomprises account information and a recommendation identifying one ofthe user's 106 accounts. In this example, the virtual assessment data111 identifies different accounts available for the user 106 andrecommends an account that provides some benefit to the user 106 if usedfor the transaction. In other examples, the server 102 generates virtualassessment data 111 comprising any other type and/or combination ofinformation. The server 102 sends the virtual assessment data 111 to theaugmented reality user device 400.

The augmented reality user device 400 receives the virtual assessmentdata 111 and processes the virtual assessment data 111 to access theinformation provided by the server 102. In one embodiment, the augmentedreality user device 400 presents account information and arecommendation identifying one of the user's 106 account as virtualobjects overlaid with tangible objects in a real scene in front of theuser 106 in real time. In other embodiments, the augmented reality userdevice 400 presents recommendations, account information, availableoffers for the user 106, and/or any other information as virtual objectsoverlaid with tangible objects in the real scene in front of the user106. Examples of the augmented reality user device 400 presentinginformation to the user 106 as virtual objects overlaid with tangibleobjects in a real scene in front of the user 106 are described in FIGS.2 and 3.

The augmented reality user device 400 determines whether there are anyoffers available for the user 106. In one embodiment, the augmentedreality user device 400 determines there are offers available for theuser 106 based on the presence of the offers in the virtual assessmentdata 111. The augmented reality user device 400 presents the offers tothe user 106 as a virtual object overlaid with tangible objects in thereal scene in front of the user 106. When the augmented reality userdevice 400 presents the one or more available offers, the augmentedreality user device 400 determines whether the user 106 selects an offerto activate. The augmented realty user device 400 receives theindication of the selected offer from the user 106 as a voice command, agesture, an interaction with a button on the augmented reality userdevice 400, or in any other suitable form. The augmented reality userdevice 400 is configured to send an activation command 128 identifyingthe selected offer to the remote server 102.

The server 102 receives the activation command 128 identifying theselected offer and facilitates the activation of the selected offer forthe user 106. For example, the server 102 exchanges messages with athird-party database 118 to activate the selected offer for the user106. In one embodiment, the server 102 sends virtual assessment data 111to the augmented reality user device 400 that indicates the selectedoffer has been activated. The activation notification may be presentedto the user 106 by the augmented reality user device 400 as a virtualobject.

FIG. 2 is an embodiment of a first person view from a display 408 of anaugmented reality user device 400 overlaying virtual objects 202 ontotangible objects 204 within a real scene 200. Examples of tangibleobjects 204 include, but are not limited to, products, buildings,floors, walls, shelves, furniture, people, or any other physicalobjects. In FIG. 2, a user 106 is shopping in a store using theaugmented reality user device 400. The user 106 has several objects 150(e.g. groceries) in a shopping cart 152. The user 106 employs theaugmented reality user device 400 to determine the price of the objects150 within the shopping cart 152, to identify one or more offersavailable to the user 106, to receive account information for the user106, to receive a recommendation based on the objects 150 within theshopping cart 152, and/or to receive any other information based on theuser 106, the location of the user 106, or the objects 150 within theshopping cart 152.

For example, the user 106 uses the augmented reality user device 400 todetermine the price of the objects 150 in the shopping cart 152. Theaugmented reality user device 400 identifies the objects 150 in theshopping cart 152 and determines a price of the objects 150 in theshopping cart 152. In one embodiment, the augmented reality user device400 queries a third-party database 118 using descriptors for the objects150 to determine the price of the objects 150. In one embodiment, theaugmented reality user device 400 presents the determined price of theobjects 150 as a virtual object 206 in the real scene 200 in front ofthe user 106 using the display 408 of the augmented reality user device400.

Additionally, the user 106 uses the augmented reality user device 400 toobtain account information for the user 106 and to determine whetherthere are any available offers for the user 106. In one embodiment, theaugmented reality user device 400 generates a token 110 comprising auser identifier 108 for the user 106, product identifiers 115 for theobjects 150, and a vendor identifier 117 identifying the location wherethe user 106 is shopping. The augmented reality user device 400 sendsthe token 110 to the remote server 102 to request account informationfor the user 106 and to identify any available offers for the user 106.The account information for the user 106 and available offers for theuser 106 may be determined based on information from multiple sources.For example, account information for the user 106 is stored in the sever102 and information about available offers is stored in one or morethird-party databases 118. In other examples, account information andinformation about available offers may be located in any other sourcesand/or combinations of sources. Tokens 110 allow the augmented realityuser device 400 to make a single request regardless of the number ofsources used to compile the requested information. Using a singlerequest improves the efficiency of the system compared to other systemsthat make individual requests to each source. Additionally, theaugmented reality user device 400 is able to request information withoutknowledge of which sources or how many sources need to be queried forthe information.

In response to sending the token 110, the augmented reality user device400 receives virtual assessment data 111 from the remote server 102. Inone embodiment, the virtual assessment data 111 comprises accountinformation for the user 106 and a recommendation for the user 106. Thevirtual assessment data 111 may also identify offers available to theuser 106. In FIG. 2, the account information includes the user's 106accounts and budget information. The augmented reality user device 400presents the user's 106 account information as a virtual object 208overlaid with the real scene 200 in front of the user 106. In thisexample, the virtual object 208 presents two accounts for the user 106.The augmented reality user device 400 presents the budget information asanother virtual object 210 overlaid with the real scene 200 in front ofthe user 106. The virtual object 210 presents the budget information toillustrate the current status of the user's 106 budget. In this example,the user's 106 accounts are shown as a list and the budget informationis shown as a pie chart. However, in other examples the user's 106accounts and the budget information may be presented in any othersuitable form. In this example, the virtual object 208 presenting theuser's 106 accounts also presents the recommendation for the user 106.For example, the virtual object 208 presents a recommendationidentifying for the user 106 to use the first account in the list ofaccounts. The augmented reality user device 400 uses any suitabletechnique to identify or present a recommendation to the user 106. Forexample, the augmented reality user device 400 presents a virtual objectwith just a recommendation.

The augmented reality user device 400 presents the available offers forthe user 106 as a virtual object 212. In this example, two offers arepresented to the user 106. Any number of offers may be presented to theuser 106. In other examples, the augmented reality user device 400presents any other information linked with the user 106, the objects150, and/or the vendor.

The following is another non-limiting example of how the augmentedreality system 100 may operate when the user 106 is looking at medium tohigh price product at a store or dealership. In this example, a user 106is using the augmented reality user device 400 in a store. The user 106authenticates themselves before using the augmented reality user device400 by providing credentials (e.g. a log-in and password) or a biometricsignal. The augmented reality user device 400 authenticates the user 106based on the user's input and allows the user 106 to generate and sendtokens 110. The augmented reality user device 400 identifies the user106 and a user identifier 108 for the user 106 upon authenticating theuser 106. Once the user 106 has been authenticated, the user identifier108 may be used by other systems and devices (e.g. remote server 102 ora third-party database 118) to identify and authenticate the user 106without requiring the user 106 to provide additional credentials foreach system.

Once the user 106 is authenticated, the augmented reality user device400 identifies the location of the user, for example, the store wherethe user 106 is shopping. In one embodiment, the augmented reality userdevice 400 identifies the location of the user 106 based on thegeographic location of the user 106 similar to as described in previousexamples. In another embodiment, the augmented reality user device 400uses object recognition and/or optical character recognition to identifythe store similar to as described in previous examples. In otherembodiments, the augmented reality user device 400 identifies the storeusing any other suitable technique.

In this example, the user 106 looks at an appliance (e.g. a washer) withthe augmented reality user device 400. The augmented reality user device400 captures an image 407 of the appliance and performs objectrecognition and/or optical character recognition on the captured image407 to identify the appliance. For example, the augmented reality userdevice 400 identifies the appliance based on the size and shape of theappliance and/or based on text or logos on its packaging.

The augmented reality user device 400 determines the price for theappliance. In one embodiment, the augmented reality user device 400accesses a third-party database 118 linked with the store to determinethe price for the appliance. For example, the augmented reality userdevice 400 queries the third-party database 118 using descriptors forthe identified appliance. In another embodiment, the augmented realityuser device 400 determines a price of the appliance by identifyingbarcodes, QR codes, digital watermarks, or price tags associated withthe appliance. For example, the augmented reality user device 400identifies a price tag on the appliance. As another example, theaugmented reality user device 400 identifies a price tag on a shelfproximate to the appliance.

The augmented reality user device 400 generates a token 110 and sendsthe token 110 to the remote server 102. In one embodiment, the augmentedreality user device 400 generates a token 110 comprising the useridentifier 108 for the user 106, a vendor identifier 117 identifying thestore where the user 106 is located, a product identifier 115identifying the appliance, and the price of the appliance. In otherembodiments, the token 110 comprises any other suitable information orcombinations of information. The augmented reality user device 400encrypts and/or encodes the token 110 prior to sending the token 110 tothe remote server 102.

The server 102 receives the token 110 and processes the token 110 toidentify the user identifier 108, the vendor identifier 117, the productidentifier 115, and the price of the appliance. The server 102 decryptsor decodes the token 110 when the token 110 is encrypted or encoded bythe augmented reality user device 400. The server 102 uses the useridentifier 108 to look-up account information and accounts for the user106 in the account information database 120.

In one embodiment, the server 102 determines whether there are any newaccounts available for the user 106 based on the user's accountinformation, the location of the user 106 (e.g. the vendor identifier117), and/or the appliance (e.g. the product identifier 115). Examplesof new accounts include, but are not limited to, credit cards, loans,lines of credit, and any other financing options. For example, theserver 102 identifies lines of credit or loans available to the user 106based on their account information (e.g. credit score). In this example,the server 102 prequalifies the user 106 for a new line a credit basedon their account information. As another example, the server 102identifies a new credit card available to the user 106 based on thelocation of the user 106 and account information for the user 106. Inthis example, the server 102 identifies new accounts for the user 106that are specific to the location of the user 106 and that the user 106qualifies for. For instance, a vendor specific loyalty program accountor a credit card account.

In one embodiment, the server 102 queries one or more third-partydatabases 118 for available new accounts based on the user's 106identity (e.g. the user identifier 108), the appliance (e.g. the productidentifiers 115), and/or the store where the user 106 is shopping (e.g.the vendor identifier 117). For instance, a third-party database 118linked with the store identified by the vendor identifier 108 andprovides information related to lines of credit accounts, financingoption accounts, and/or loyalty program accounts.

In one embodiment, the server 102 obtains information related to othermodels or alternatives for the appliance. For example, the server 102uses the product identifier 115 to identify other similar appliances inthe product information database 123. As another example, the sever 102sends a data request 126 with the product identifier 115 to athird-party database 118 to request information for other similarappliances. The server 102 receives information about other similarappliances based on the product identifier 115. The product informationabout other similar appliances may identify alternative appliancesand/or alternative vendors for the appliance.

In one embodiment, the server 102 obtains product information linkedwith the appliance. For example, the server 102 uses the productidentifier 115 to identify product information linked with the appliancein the product information database 123. As another example, the server102 sends a data request 126 with the product identifier 115 to requestproduct information linked with the appliance. The product informationcomprises product reviews, product descriptions, warranty information,historical pricing information, seasonal pricing information, secondarycosts (e.g. insurance costs and maintenance costs), demographicinformation about other people who purchased the appliance, and/or anyother information linked with the appliance.

The server 102 generates virtual assessment data 111 that comprisesaccount information for the user 106, one or more available new accountsfor the user 106, and/or any other information. For example, the server102 generates virtual assessment data 111 that comprises the accountinformation and one or more available new accounts for the user 106. Inthis example, the virtual assessment data 111 identifies differentexisting accounts available for the user 106 and one or more availablenew accounts the user 106 may be interested in. As another example, theserver 102 generates virtual assessment data 111 that comprises accountinformation for the user 106 and product information for the appliance.In this example, the virtual assessment data 111 identifies differentaccounts available for the user 106 and provides information related tothe appliance. As another example, the server 102 generates virtualassessment data 111 that comprises account information and identifiesalternative products from other vendors. In this example, the virtualassessment data 111 identifies different accounts available for the user106 and provides information to the user 106 about other similarproducts the user's 106 might want to consider. In other examples, theserver 102 generates virtual assessment data 111 comprising any otherinformation and/or combination of information. The server 102 then sendsthe virtual assessment data 111 to the augmented reality user device400.

The augmented reality user device 400 receives the virtual assessmentdata 111 and processes the virtual assessment data 111 to access theinformation provided by the server 102. In one embodiment, the augmentedreality user device 400 presents account information and new availableaccounts for the user 106 as virtual objects overlaid with tangibleobjects in a real scene in front of the user 106 in real time. In otherembodiments, the augmented reality user device 400 presentsrecommendations, account information, product information, availableoffers for the user 106, and/or any other information as virtual objectsoverlaid with tangible objects in the real scene.

The augmented reality user device 400 determines whether there are anynew accounts available for the user 106. In one embodiment, theaugmented reality user device 400 determines there are new accountsavailable for the user 106 based on the presence of information linkedwith new accounts in the virtual assessment data 111. The augmentedreality user device 400 presents the new accounts available to the user106 as a virtual object overlaid with tangible objects in the real scenein front of the user 106. When the augmented reality user device 400presents the one or more available new accounts, the augmented realityuser device 400 determines whether the user 106 selects a new account toactivate. The augmented realty user device 400 receives the indicationof the selected new account from the user 106 as a voice command, agesture, an interaction with a button on the augmented reality userdevice 400, or in any other suitable form. The augmented reality userdevice 400 is configured to send an activation command 128 identifyingthe selected new account to the remote server 102. The augmented realityuser device 400 allows the user 106 to quickly identify any new accountsthe user 106 is prequalified for based on their personal informationwithout the user 106 having to manually search for and apply fordifferent accounts. The augmented reality user device 400 also providesthe ability activate one of the new accounts using previously storedaccount information and without the user 106 having to provide all oftheir personal information for each account they would like to activate.

The server 102 receives the activation command 128 identifying theselected new account and facilitates activating the selected new accountfor the user 106. For example, the server 102 exchanges messages with athird-party database 118 to activate the selected new account for theuser 106. The server 102 uses account information for the user 106 orany other to activate the new account. For instance, the server 102 usescredit information and personal information for the user 106 to activatethe new account. In one embodiment, the server 102 sends virtualassessment data 111 to the augmented reality user device 400 thatindicates the selected new account has been activated. The activationnotification may be presented to the user 106 by the augmented realityuser device 400 as a virtual object.

FIG. 3 is another embodiment of a first person view from a display 408of an augmented reality user device 400 overlaying virtual objects 202with a real scene 200. In FIG. 3, the user 106 is shopping in a storeusing the augmented reality user device 400. The user 106 is looking anappliance 310. The user 106 employs the augmented reality user device400 to determine the price of the appliance 310, to identify one or moreavailable new accounts for the user 106, to receive account informationfor the user 106, to receive product information based on the appliance310, and/or to receive any other information based on the user 106, thelocation of the user 106, or the appliance 310.

The user 106 uses the augmented reality user device 400 to determine theprice of the appliance 310. In one embodiment, the augmented realityuser device 400 queries a third-party database 118 using descriptors forthe appliance 310 to determine the price of the appliance 310. In oneembodiment, the augmented reality user device 400 presents thedetermined price of the objects 150 as a virtual object 202 in the realscene 200 using the display 408 of the augmented reality user device400.

Additionally, the user 106 uses the augmented reality user device 400 toobtain account information for the user 106 and to determine whetherthere are any available new accounts for the user 106. In oneembodiment, the augmented reality user device 400 generates a token 110comprising a user identifier 108 for the user 106, a product identifier115 for the appliance 310, and a vendor identifier 117 identifying thelocation where the user 106 is shopping. The augmented reality userdevice 400 sends the token 110 to the remote server 102 to requestaccount information for the user 106 and to identify any available newaccounts for the user 106. The account information for the user 106 andavailable new accounts for the user 106 may be determined based oninformation from multiple sources. For example, account information forthe user 106 is stored in the sever 102 and information about availablenew accounts is stored in one or more third-party databases 118. Inother examples, account information and information about available newaccounts may be located in any other sources and/or combinations ofsources. Tokens 110 allow the augmented reality user device 400 torequest information regardless of the number of sources used to compilethe requested information. Using a reduced number of requests improvesthe efficiency of the system compared to other systems that makeindividual requests to each source. Additionally, the augmented realityuser device 400 is able to request information without knowledge ofwhich sources or how many sources need to be queried for theinformation.

In response to sending the token 110, the augmented reality user device400 received virtual assessment data 111 from the remote server 102. Inone embodiment, the virtual assessment data 111 comprises accountinformation, available new accounts, and a recommendation for the user106. The augmented reality user device 400 presents the user's 106account information as a virtual object 302 overlaid with the real scene200 in front of the user 106. In this example, the virtual object 302presents two accounts for the user 106. The augmented reality userdevice 400 presents the available new accounts as another virtual object304 overlaid with the real scene 200 in front of the user 106. In thisexample, the user's 106 existing accounts and available new accounts areeach shown as lists. However, in other examples the user's 106 existingaccounts and available new accounts may be presented in any othersuitable form.

The augmented reality user device 400 presents the recommendation forthe user 106 as another virtual object 306 overlaid with the real scene200 in front of the user 106. In FIG. 3, the recommendation identifies avendor to purchase the appliance 310 from and one of the user's 106accounts to make the purchase. The recommendation may comprise any otherinformation or combination of information for the user 106.

The augmented reality user device 400 presents alternative vendors ofthe appliance 310 for the user 106 as a virtual object 308. In thisexample, three different vendors and their prices for the appliance 310are presented to the user 106. In other examples, any number vendors maybe presented to the user 106. In other examples, the augmented realityuser device 400 presents any other information linked with the user 106,the appliance 310, and/or the vendor.

FIG. 4 is a schematic diagram of an embodiment of an augmented realityuser device 400 employed by an augmented reality system 100. Theaugmented reality user device 400 is configured to capture an image 407of an object (e.g. a product, a sign, or a business), to send a token110 identifying a user 106 and the object to a remote server 102, toreceive virtual assessment data 111 in response to sending the token110, and to present virtual objects overlaid onto one or more tangibleobjects in a real scene based on the information provided by the virtualassessment data 111. Examples of the augmented reality user device 400in operation is described in FIGS. 5, 7, and 10.

The augmented reality user device 400 comprises a processor 402, amemory 404, a camera 406, a display 408, a wireless communicationinterface 410, a network interface 412, a microphone 414, a GPS sensor416, and one or more biometric devices 418. The augmented reality userdevice 400 may be configured as shown or in any other suitableconfiguration. For example, augmented reality user device 400 maycomprise one or more additional components and/or one or more showncomponents may be omitted.

Examples of the camera 406 include, but are not limited to,charge-coupled device (CCD) cameras and complementary metal-oxidesemiconductor (CMOS) cameras. The camera 406 is configured to captureimages 407 of people, text, and objects within a real environment. Thecamera 406 is configured to capture images 407 continuously, atpredetermined intervals, or on-demand. For example, the camera 406 isconfigured to receive a command from a user to capture an image 407. Inanother example, the camera 406 is configured to continuously captureimages 407 to form a video stream of images 407. The camera 406 isoperable coupled to an object recognition engine 424, an opticalcharacter (OCR) recognition engine 426, and/or a gesture recognitionengine 428 and provides images 407 to the object recognition engine 424,the OCR recognition engine 426, and/or the gesture recognition engine428 for processing, for example, to identify gestures, text, and/orobjects in front of the user.

The display 408 is configured to present visual information to a user inan augmented reality environment that overlays virtual or graphicalobjects onto tangible objects in a real scene in real-time. In anembodiment, the display 408 is a wearable optical head-mounted displayconfigured to reflect projected images and allows a user to see throughthe display 408. For example, the display 408 may comprise displayunits, lens, semi-transparent mirrors embedded in an eye glassstructure, a contact lens structure, a visor structure, or a helmetstructure. Examples of display units include, but are not limited to, acathode ray tube (CRT) display, a liquid crystal display (LCD), a liquidcrystal on silicon (LCOS) display, a light emitting diode (LED) display,an active matric OLED (AMOLED), an organic LED (OLED) display, aprojector display, or any other suitable type of display as would beappreciated by one of ordinary skill in the art upon viewing thisdisclosure. In another embodiment, the display 408 is a graphicaldisplay on a user device. For example, the graphical display may be thedisplay of a tablet or smart phone configured to display an augmentedreality environment with virtual or graphical objects overlaid ontotangible objects in a real scene in real-time.

Examples of the wireless communication interface 410 include, but arenot limited to, a Bluetooth interface, a radio frequency identifier(RFID) interface, a near-field communication (NFC) interface, a localarea network (LAN) interface, a personal area network (PAN) interface, awide area network (WAN) interface, a Wi-Fi interface, a ZigBeeinterface, or any other suitable wireless communication interface aswould be appreciated by one of ordinary skill in the art upon viewingthis disclosure. The wireless communication interface 410 is configuredto allow the processor 402 to communicate with other devices. Forexample, the wireless communication interface 410 is configured to allowthe processor 402 to send and receive signals with other devices for theuser (e.g. a mobile phone) and/or with devices for other people. Thewireless communication interface 410 is configured to employ anysuitable communication protocol.

The network interface 412 is configured to enable wired and/or wirelesscommunications and to communicate data through a network, system, and/ordomain. For example, the network interface 412 is configured forcommunication with a modem, a switch, a router, a bridge, a server, or aclient. The processor 402 is configured to receive data using networkinterface 412 from a network or a remote source.

Microphone 414 is configured to capture audio signals (e.g. voicecommands) from a user and/or other people near the user. The microphone414 is configured to capture audio signals continuously, atpredetermined intervals, or on-demand. The microphone 414 is operablycoupled to the voice recognition engine 422 and provides captured audiosignals to the voice recognition engine 422 for processing, for example,to identify a voice command from the user.

The GPS sensor 416 is configured to capture and to provide geographicallocation information. For example, the GPS sensor 416 is configured toprovide the geographic location of a user employing the augmentedreality user device 400. The GPS sensor 416 is configured to provide thegeographic location information as a relative geographic location or anabsolute geographic location. The GPS sensor 416 provides the geographiclocation information using geographic coordinates (i.e. longitude andlatitude) or any other suitable coordinate system.

Examples of biometric devices 418 include, but are not limited to,retina scanners and finger print scanners. Biometric devices 418 areconfigured to capture information about a person's physicalcharacteristics and to output a biometric signal 431 based on capturedinformation. A biometric signal 431 is a signal that is uniquely linkedto a person based on their physical characteristics. For example, abiometric device 418 may be configured to perform a retinal scan of theuser's eye and to generate a biometric signal 431 for the user based onthe retinal scan. As another example, a biometric device 418 isconfigured to perform a fingerprint scan of the user's finger and togenerate a biometric signal 431 for the user based on the fingerprintscan. The biometric signal 431 is used by a biometric engine 432 toidentify and/or authenticate a person. In one embodiment, the biometricdevice 418 are configured to collect health information (e.g. vitals)for a user as biometric signals 431. Examples of health informationincludes, but is not limited to, heart rate, blood sugar, eye dilation,and perspiration levels.

The processor 402 is implemented as one or more CPU chips, logic units,cores (e.g. a multi-core processor), FPGAs, ASICs, or DSPs. Theprocessor 402 is communicatively coupled to and in signal communicationwith the memory 404, the camera 406, the display 408, the wirelesscommunication interface 410, the network interface 412, the microphone414, the GPS sensor 416, and the biometric devices 418. The processor402 is configured to receive and transmit electrical signals among oneor more of the memory 404, the camera 406, the display 408, the wirelesscommunication interface 410, the network interface 412, the microphone414, the GPS sensor 416, and the biometric devices 418. The electricalsignals are used to send and receive data (e.g. images 407 and tokens110) and/or to control or communicate with other devices. For example,the processor 402 transmit electrical signals to operate the camera 406.The processor 402 may be operably coupled to one or more other devices(not shown).

The processor 402 is configured to process data and may be implementedin hardware or software. The processor 402 is configured to implementvarious instructions. For example, the processor 402 is configured toimplement a virtual overlay engine 420, a voice recognition engine 422,an object recognition engine 424, an OCR recognition engine 426, agesture recognition engine 428, a virtual assessment engine 430, and abiometric engine 432. In an embodiment, the virtual overlay engine 420,the voice recognition engine 422, the object recognition engine 424, theOCR recognition engine 426, the gesture recognition engine 428, thevirtual assessment engine 430, and the biometric engine 432 areimplemented using logic units, FPGAs, ASICs, DSPs, or any other suitablehardware.

The virtual overlay engine 420 is configured to overlay virtual objectsonto tangible objects in a real scene using the display 408. Forexample, the display 408 may be head-mounted display that allows a userto simultaneously view tangible objects in a real scene and virtualobjects. The virtual overlay engine 420 is configured to process data tobe presented to a user as an augmented reality virtual object on thedisplay 408. Examples of overlaying virtual objects onto tangibleobjects in a real scene is shown in FIGS. 2, 3, and 9.

The voice recognition engine 422 is configured to capture and/oridentify voice patterns using the microphone 414. For example, the voicerecognition engine 422 is configured to capture a voice signal from aperson and to compare the captured voice signal to known voice patternsor commands to identify the person and/or commands provided by theperson. For instance, the voice recognition engine 422 is configured toreceive a voice signal to authenticate a user and/or to identify aselected option or an action indicated by the user.

The object recognition engine 424 is configured to identify objects,object features, branding, text, and/or logos using images 407 or videostreams created from a series of images 407. In one embodiment, theobject recognition engine 424 is configured to identify objects and/ortext within an image 407 captured by the camera 406. In anotherembodiment, the object recognition engine 424 is configured to identifyobjects and/or text in about real-time on a video stream captured by thecamera 406 when the camera 406 is configured to continuously captureimages 407. The object recognition engine 424 employs any suitabletechnique for implementing object and/or text recognition as would beappreciated by one of ordinary skill in the art upon viewing thisdisclosure.

The OCR recognition engine 426 is configured to identify objects, objectfeatures, text, and/or logos using images 407 or video streams createdfrom a series of images 407. In one embodiment, the OCR recognitionengine 426 is configured to identify objects and/or text within an image407 captured by the camera 306. In another embodiment, the OCRrecognition engine 426 is configured to identify objects and/or text inabout real-time on a video stream captured by the camera 406 when thecamera 406 is configured to continuously capture images 407. The OCRrecognition engine 426 employs any suitable technique for implementingobject and/or text recognition as would be appreciated by one ofordinary skill in the art upon viewing this disclosure.

The gesture recognition engine 428 is configured to identify gesturesperformed by a user and/or other people. Examples of gestures include,but are not limited to, hand movements, hand positions, fingermovements, head movements, and/or any other actions that provide avisual signal from a person. For example, gesture recognition engine 428is configured to identify hand gestures provided by a user to indicatevarious commands such as a command to initiate a request for anaugmented reality overlay for an object. The gesture recognition engine428 employs any suitable technique for implementing gesture recognitionas would be appreciated by one of ordinary skill in the art upon viewingthis disclosure.

The virtual assessment engine 430 is configured to identify the locationof the user 106 and to generate a vendor identifier 117 or a locationidentifier 902 based on the location of the user 106. In one embodiment,the augmented reality user device 400 identifies the location of theuser 106 based on the geographic location of the user 106. For example,the augmented reality user device 400 uses geographic locationinformation provided by the GPS sensor 416 with a map database (e.g. athird-party database 118) to determine the location of the user 106. Inanother embodiment, the augmented reality user device 400 is configuredto use object recognition and/or optical character recognition toidentify the location of the user 106. For example, the augmentedreality user device 400 is configured to identify the location of theuser 106 based on identifying buildings, structures, landmarks,branding, signs, logo, and/or any other types of objects around the user106. In other embodiments, the augmented reality user device 400determines the location of the user 106 based on any other informationor using any other suitable technique as would be appreciated by one ofordinary skill in the art.

The virtual assessment engine 430 is configured to identify tangibleobjects 150 in front of the user 106 and to generate product identifiers115 for the objects 150. The virtual assessment engine 430 is configuredto use object recognition and/or optical character recognition toidentify an object 150. In one embodiment, the virtual assessment engine430 is configured to capture an image 407 of an object 150 and toperform object recognition and/or optical character recognition on theimage 407 of the object 150 to identify the object 150. The virtualassessment engine 430 is configured to identify the object 150 based onthe size, shape, features, and/or any other characteristics of theobject 150. For example, the virtual assessment engine 430 identifiesthe object 150 based on branding, text, or logos on the object 150 orits packaging. The virtual assessment engine 430 identifies the object150 based on any other characteristics of the object 150 or using anyother suitable technique as would be appreciated by one of ordinaryskill in the art.

The virtual assessment engine 430 is further configured to determine aprice for one or more objects 150. In one embodiment, the virtualassessment engine 430 is configured to access a third-party database 118to determine the price for each of the objects 150. For example, thevirtual assessment engine 430 queries a third-party database 118 linkedwith a vendor of the one or more objects 150 to determine the price ofthe one or more objects 150. In one embodiment, the virtual assessmentengine 430 sends a message 113 with descriptors for the one or moreobjects 150 to the third-party database 118.

In another embodiment, the virtual assessment engine 430 determines theprice of the objects 150 using barcodes, QR codes, digital watermarks,or price tags associated with the objects 150. For example, the virtualassessment engine 430 identifies price tags on the objects 150 anddetermines the price of the objects 150 based on the price tags. Asanother example, the virtual assessment engine 430 identifies price tagson the shelves when the user 106 picks up an object 150. In anotherembodiment, the virtual assessment engine 430 determines the total priceof the objects 150 based on an input provided by the user 106. Thevirtual assessment engine 430 is configured to determine prices of theobjects 150 based on a voice command, a gesture, an interaction with abutton on the augmented reality user device 400, or in any form of userinput. For example, the virtual assessment engine 430 is configured toreceive voice commands from the user 106 stating the prices of each ofthe objects 150. In other examples, the virtual assessment engine 430 isconfigured to determine the price of the objects 150 using any othersuitable technique as would be appreciated by one of ordinary skill inthe art.

The virtual assessment engine 430 is configured to collect user historydata for a user 106. Examples of user history data include, but are notlimited to, location history, internet search history, transactionhistory, biometric signal history, and/or any other kind of history forthe user 106. In one embodiment, the virtual assessment engine 430 isconfigured to collect user history data from one or more other devicessuch as a mobile device of the user or a third-party database 118. Inother embodiments, the virtual assessment engine 430 is configured tocollect user history data from any suitable sources.

The virtual assessment engine 430 is configured to generate a token 110for requesting information for the user 106. In one embodiment, thetoken 110 comprises a user identifier 108, a vendor identifier 117, oneor more product identifiers 115, and a price 109. In other embodiments,the token 110 identifies any other information or combination ofinformation. The virtual assessment engine 430 is further configured toencrypt and/or encode the token 110. Encrypting and encoding the token110 obfuscates and mask information being communicated by the token 110.Masking the information being communicated protects users and theirinformation in the event of unauthorized access to the network and/ordata occurs. The virtual assessment engine 430 employs any suitableencryption or encoding technique as would be appreciated by one ofordinary skill in the art.

The virtual assessment engine 430 is configured to send the token 110 toa remote server 102 as a data request to initiate the process ofobtaining information for the user 106. The virtual assessment engine430 is further configured to provide the information (e.g. virtualoverlay data 111) received from the remote server 102 to the virtualoverlay engine 420 to present the information as one or more virtualobjects overlaid with tangible objects in a real scene. Examples ofemploying the virtual assessment engine 430 to request information andpresent the information to a user 106 is described in FIGS. 5, 7, and10.

The virtual assessment engine 430 is further configured to employ thevirtual overlay engine 420 to present one or more offers or new accountsthat are available for the user 106, geolocation notifications 908,and/or any other information. In one embodiment, the virtual assessmentengine 430 is configured to identify selected offers and new accounts bythe user 106. For example, the virtual assessment engine 430 isconfigured to identify a selected offer and to send an activationcommand 128 to the remote server 102 that identifies the selected offerto activate. As another example, the virtual assessment engine 430 isconfigured to identify a selected new account for the user 106 and tosend an activation command 128 to the remote server 102 that identifiesthe selected new account to activate. The user 106 may identify aselection by giving a voice command, performing a gesture, interactingwith a physical component (e.g. a button, knob, or slider) of theaugmented reality user device 400, or any other suitable mechanism aswould be appreciated by one of ordinary skill in the art. An example ofemploying the virtual assessment engine 430 to identify a selected offerand to send an activation command 128 to the remote server 102 thatidentifies the selected offer is described in FIG. 5. An example ofemploying the virtual assessment engine 430 to identify a selected newaccount and to send an activation command 128 to the remote server 102that identifies the selected offer is described in FIG. 7.

The biometric engine 432 is configured to identify a person based on abiometric signal 431 generated from the person's physicalcharacteristics. The biometric engine 432 employs one or more biometricdevices 418 to identify a user 106 based on one or more biometricsignals 418. For example, the biometric engine 432 receives a biometricsignal 431 from the biometric device 418 in response to a retinal scanof the user's eye and/or a fingerprint scan of the user's finger. Thebiometric engine 432 compares biometric signals 431 from the biometricdevice 418 to previously stored biometric signals 431 for the user 106to authenticate the user 106. The biometric engine 432 authenticates theuser 106 when the biometric signals 431 from the biometric devices 418substantially matches (e.g. is the same as) the previously storedbiometric signals 431 for the user 106. In one embodiment, the biometricengine 432 is configured to employ biometric device 418 to collecthealth information (e.g. vitals) for a user 106.

The memory 404 comprise one or more disks, tape drives, or solid-statedrives, and may be used as an over-flow data storage device, to storeprograms when such programs are selected for execution, and to storeinstructions and data that are read during program execution. The memory404 may be volatile or non-volatile and may comprise ROM, RAM, TCAM,DRAM, and SRAM. The memory 404 is operable to store images 407, tokens110, biometric signals 431, virtual overlay instructions 434, voicerecognition instructions 436, OCR recognition instructions 438, objectrecognition instructions 440, gesture recognition instructions 442,virtual assessment instructions 444, biometric instructions 446, and anyother data or instructions.

Images 407 comprises images captured by the camera 406 and images 407from other sources. In one embodiment, images 407 comprise images usedby the augmented reality user device 400 when performing objectrecognition and/or optical character recognition. Images 407 can becaptured using camera 406 or downloaded from another source such as aflash memory device or a remote server via an Internet connection.

Biometric signals 431 are signals or data that is generated by abiometric device 418 based on a person's physical characteristics.Biometric signals 431 are used by the augmented reality user device 400to identify and/or authenticate an augmented reality user device 400user by comparing biometric signals 431 captured by the biometricdevices 418 with previously stored biometric signals 431.

Tokens 110 are generated by the virtual assessment engine 430 and sentto a remote server 102 to initiate a process for obtaining informationlinked with an object, a vendor, and/or a user 106. Tokens 110 compriseany suitable information for requesting information from the remoteserver 102 and/or one or more other sources (e.g. third-party databases118). In one embodiment, the token 110 is a message or data requestcomprising information identifying a user 106 (e.g. user identifier108), one or more objects 150 (e.g. one or more product identifiers115), and the location of the user 106 (e.g. a vendor identifier 117 ora location identifier 902). In other embodiments, the token 110comprises any other information or combinations of information. Examplesof the augmented reality user device 400 generating and sending a token110 to initiate a process for obtaining information is described inFIGS. 5, 7, and 10.

The virtual overlay instructions 434, the voice recognition instructions436, the OCR recognition instructions 438, the object recognition engine440, the gesture recognition instructions 442, the virtual assessmentinstructions 444, and the biometric instructions 446 each comprise anysuitable set of instructions, logic, rules, or code operable to executethe virtual overlay engine 420, the voice recognition engine 422, theOCR recognition engine 426, the object recognition engine 424, thegesture recognition engine 428, the virtual assessment engine 430, andthe biometric engine 432, respectively.

FIG. 5 is a flowchart of an embodiment of an augmented realityoverlaying method 500 for an augmented reality system 100. In oneembodiment, the augmented reality system 100 is configured similar tothe augmented reality system 100 described in FIG. 1. Method 500 isemployed by the processor 402 of the augmented reality user device 400to generate a token 110 based on a user 106 of the augmented realityuser device 400, the location of the user 106, and one or more products.The augmented reality user device 400 uses the token 110 to requestaccount information for the user 106 and/or to determine whether thereare any offers available for the user 106. The augmented reality userdevice 400 receives the requested information in response to sending thetoken 110 and presents received information as virtual objects overlaidwith tangible object in a real scene in front of the user 106.

At step 502, the augmented reality user device 400 authenticates a user106. The user 106 authenticates themselves by providing credentials(e.g. a log-in and password) or a biometric signal. The augmentedreality user device 400 authenticates the user 106 based on the user'sinput. The user 106 is able to generate and send tokens 110 using theaugmented reality user device 400 upon authenticating the user 106.

At step 504, the augmented reality user device 400 identifies a useridentifier 108 for the user 106. Once the user 106 has beenauthenticated, the augmented reality user device 400 identifies the user106 and a user identifier 108 for the user 106. The user identifier 108may be used to identify and authenticate the user 106 in other systems,for example, third-party databases 118.

At step 506, the augmented reality user device 400 identifies a vendorbased on the geographic location of the user 106. In one embodiment, theaugmented reality user device 400 uses geographic location informationprovided by the GPS sensor 416 with a map database to determine thelocation of the user 106 and to identify the vender where the user 106is shopping. In another embodiment, the augmented reality user device400 uses object recognition and/or optical character recognition toidentify the vendor based on structures, branding, signs, and/or logos.

At step 508, the augmented reality user device 400 captures an image 407of products in a cart. In one embodiment, the user 106 provides acommand or signal to the augmented reality user device 400 that triggersthe camera 406 to capture an image 407 of the products in the cart. Inanother embodiment, the augmented reality user device 400 and the camera406 are configured to continuously or periodically capture images 407.

At step 510, the augmented reality user device 400 performs objectrecognition on the image 407 to identify the products in the cart. Forexample, the augmented reality user device 400 identifies the productsbased on the size and shape of the products and/or based on branding,text, or logos on their packaging. In other examples, the augmentedreality user device 400 identifies the products based on any otherfeatures of the products and/or using any other suitable technique.

At step 512, the augmented reality user device 400 determines a pricefor the products in the cart. In one embodiment, the augmented realityuser device 400 queries a third-party database 118 using descriptors forthe identified products. In another embodiment, the augmented realityuser device 400 determines the price for the products by identifyingbarcodes, QR codes, digital watermarks, and/or price tags for theproducts. In some embodiments, step 512 is optional and may be omitted.

At step 514, the augmented reality user device 400 generates a token110. In one embodiment, the token 110 comprises the user identifier 108,product identifiers 115 linked with the products, and a vendoridentifier 117 linked with the vendor. In other embodiments, the token110 comprises the price of the products and/or any other information. Atstep 516, the augmented reality user device 400 sends the token 110 to aremote server 102.

At step 518, the augmented reality user device 400 receives virtualassessment data 111 in response to sending the token 110 to the server102. The virtual assessment data 111 comprises account information,recommendations, budget information, one or more offers available to theuser 106, any other information, and/or combinations of information.

At step 520, the augmented reality user device 400 presents arecommendation for the user 106 as a virtual object overlaid with theproducts in the real scene in front of the user 106. The augmentedreality user device 400 presents a recommendation and any otherinformation provided by the virtual assessment data 111 as virtualobjects overlaid on top of the products or adjacent to the products.When the augmented reality user device 400 presents the virtual objectsto the user 106, the user 106 can readily see their current accountinformation.

At step 522, the augmented reality user device 400 determines whetherthere are any offers available for the user 106. For example, theaugmented reality user device 400 determines that there are offersavailable to the user 106 based on the presence of the offers in thevirtual assessment data 111. In other examples, the augmented realityuser device 400 employs any other technique for determining whetherthere are offers available to the user 106. The augmented reality userdevice 400 proceeds to step 524 when the augmented reality user device400 determines that there are offers available for the user 106. Theaugmented reality user device 400 may terminate method 500 when theaugmented reality user device 400 determines there are no offersavailable for the user 106.

At step 524, the augmented reality user device 400 presents the offersavailable for the user 106 as a virtual object in the real scene infront of the user 106. The augmented reality user device 400 presentsthe offers available for the user 106 as virtual objects overlaid on topof the products or adjacent to the products. When the augmented realityuser device 400 presents the virtual objects to the user 106, the user106 can quickly identify any offer that are available to the user 106.

At step 526, the augmented reality user device 400 determines whetherthe user 106 accepts any of the offers. The augmented reality userdevice 400 determines that the user 106 accepts an offer when theaugmented reality user device 400 receives an indication of a selectedoffer from the user 106 as a voice command, a gesture, an interactionwith a button on the augmented reality user device, or in any othersuitable form. The augmented reality user device 400 proceeds to step528 when the augmented reality user device 400 determines that the user106 accepts one of the offers available for the user 106. The augmentedreality user device 400 may terminate method 500 when the augmentedreality user device 400 determines that the user 106 does not accept anyof the offers available for the user 106.

At step 528, the augmented reality user device 400 identifies theselected offer from the user 106 based on the indication provided by theuser 106. At step 530, the augmented reality user device 400 sends anactivation command 128 identifying the selected offer to the remoteserver 102.

FIG. 6 is a flowchart of another embodiment of an augmented realityoverlaying method 600 for an augmented reality system 100. In oneembodiment, the augmented reality system 100 is configured similar tothe augmented reality system 100 described in FIG. 1. Method 600 isemployed by the product analysis engine 116 in the server 102 toidentify account information and/or offers available for a user 106 ofan augmented reality user device 400 and provides the requestedinformation to the user 106 in response to receiving a token 110 fromthe augmented reality user device 400.

At step 602, the product analysis engine 116 receives a token 110 fromthe augmented reality user device 400. The product analysis engine 116decrypts and/or decodes the token 110 when the token 110 is encrypted orencoded by the augmented reality user device 400. In one embodiment, theproduct analysis engine 116 processes the token 110 to identify a useridentifier 108, one or more product identifiers 115, and a vendoridentifier 117. In other embodiments, the product analysis engine 116processes the token 110 to identify a price for one or more productslinked with the product identifiers 115 and/or any other information.

At step 604, the product analysis engine 116 determines accountinformation for a user 106 based on the user identifier 108. Forexample, the product analysis engine 116 uses the user identifier 108 tolook-up the account information and accounts for the user 106 in theaccount information database 120.

At step 606, the product analysis engine 116 determines whether thereare any offers available based on the product identifiers 115, thevendor identifier 117, and the account information for the user 106. Forexample, the product analysis engine 116 queries the offers database 122for any available offers using the user identifier 108, the accountinformation for the user 106, the product identifiers 115, and/or thevendor identifier 117. As another example, the product analysis engine116 sends a data request 126 to one or more third-party databases 118 toquery the third-party databases 118 for available offers for the user106 based on the user identifier 108, the account information for theuser 106, the product identifiers 115, and/or the vendor identifier 117.

At step 608, the product analysis engine 116 proceeds to step 610 whenthe product analysis engine 116 determines there are no offers availablefor the user 106. Otherwise, the product analysis engine 116 proceeds tostep 612 when the product analysis engine 116 determines there areoffers available for the user 106.

At step 610, the product analysis engine 116 generates a recommendationbased on the account information for the user 106. For example, theproduct analysis engine 116 generates a recommendation that identifiesone of the user's 106 accounts for a transaction to purchase theproducts. In this example, the product analysis engine 116 may selectone of the user's 106 accounts that does not cause the user 106 to goover budget or to overdraft. In other examples, the product analysisengine 116 generates any other type of recommendation based on any othercriteria. At step 614, the product analysis engine 116 generates virtualassessment data 111 comprising the recommendation and accountinformation for the user 106. The account information comprises one ormore accounts for the user 106, budget information for the user 106,and/or any other account information for the user 106.

Returning to step 608, the product analysis engine 116 proceeds to step612 when the product analysis engine 116 determines there are offersavailable. At step 612, the product analysis engine 116 generates arecommendation based on the account information for the user 106 and theavailable offers for the user 106. At step 616, the product analysisengine 116 generates virtual assessment data 111 comprising therecommendation, account information for the user 106, and the availableoffers for the user 106. For example, the product analysis engine 116generates a recommendation that identifies one or more offers (e.g.coupons) that can applied to a transaction to purchase the productsidentified by the product identifiers 115. At step 618, the productanalysis engine 116 sends the virtual assessment data 111 to theaugmented reality user device 400.

At step 620, the product analysis engine 116 determines whether theproduct analysis engine 116 has received an activation command 128 fromthe augmented reality user device 400. The product analysis engine 116proceeds to step 622 when the product analysis engine 116 receives anactivation command 128 from the augmented reality user device 400.Otherwise, the product analysis engine 116 terminates method 600 whenthe product analysis engine 116 does not receive an activation command128 from the augmented reality user device 400.

At step 622, the product analysis engine 116 activates an offer selectedby the user 106. The received activation command 128 identifies aselected offer for the user 106. The product analysis engine 116facilitates activating the selected offer. For example, the productanalysis engine 116 exchanges messages with a third-party database 118to activate the selected offer. As another example, the product analysisengine 116 notifies the offers database 122 to activate the selectedoffer. The product analysis engine 116 may employ any other suitabletechnique for activating the selected offer.

FIG. 7 is a flowchart of another embodiment of an augmented realityoverlaying method 700 for an augmented reality system 100. In oneembodiment, the augmented reality system 100 is configured similar tothe augmented reality system 100 described in FIG. 1. Method 700 isemployed by the processor 402 of the augmented reality user device 400to generate a token 110 based on a user 106 of the augmented realityuser device 400, the location of the user 106, and a product. In oneembodiment, the product is a product with a medium to high price, forexample, televisions, golf clubs, washers, dryers, appliances, vehicles,and computers. The augmented reality user device 400 uses the token 110to request account information for the user 106 and/or to determinewhether there are any new accounts available for the user 106. Theaugmented reality user device 400 receives the requested information inresponse to sending the token 110 and presents the received informationas virtual objects overlaid with tangible object in a real scene infront of the user 106.

At step 702, the augmented reality user device 400 authenticates a user106. The user 106 authenticates themselves by providing credentials(e.g. a log-in and password) or a biometric signal. The augmentedreality user device 400 authenticates the user 106 based on the user'sinput. The user 106 is able to generate and send tokens 110 using theaugmented reality user device 400 upon authenticating the user 106.

At step 704, the augmented reality user device 400 identifies a useridentifier 108 for the user 106. Once the user 106 has beenauthenticated, the augmented reality user device 400 identifies the user106 and a user identifier 108 for the user 106. The user identifier 108may be used to identify and authenticate the user 106 in other systems.

At step 706, the augmented reality user device 400 identifies a vendorbased on the geographic location of the user 106. In one embodiment, theaugmented reality user device 400 uses geographic location informationprovided by the GPS sensor 416 and map database to determine thelocation of the user 106 and to identify the vender where the user 106is shopping. In another embodiment, the augmented reality user device400 uses object recognition and/or optical character recognition toidentify the vendor based on structures, branding, signs, and/or logos.

At step 708, the augmented reality user device 400 captures an image 407of a product. In one embodiment, the user 106 provides a command orsignal to the augmented reality user device 400 that triggers the camera406 to capture an image 407 of the product. In another embodiment, theaugmented reality user device 400 and the camera 406 are configured tocontinuously or periodically capture images 407.

At step 710, the augmented reality user device 400 performs objectrecognition on the image 407 to identify the product. For example, theaugmented reality user device 400 identifies the product based on thesize and shape of the product and/or based on text or logos on itspackaging. In other examples, the augmented reality user device 400identifies the product based on any other features of the product and/orusing any other suitable technique.

At step 712, the augmented reality user device 400 determines a price ofthe product. In one embodiment, the augmented reality user device 400queries a third-party database 118 using descriptors for the identifiedproduct. In another embodiment, the augmented reality user device 400determines the price for the product by identifying a barcode, QR codes,digital watermarks, and/or price tag for the product. In someembodiments, step 712 is optional and may be omitted.

At step 714, the augmented reality user device 400 generates a token110. In one embodiment, the token 110 comprises the user identifier 108,a product identifier 115 linked with the product, and a vendoridentifier 117. In other embodiments, the token 110 comprises the priceof the product and/or any other information. At step 716, the augmentedreality user device 400 sends the token 110 to a remote server 102.

At step 718, the augmented reality user device 400 receives virtualassessment data 111. The virtual assessment data 111 comprises accountinformation, recommendations, one or more new accounts available for theuser 106, any other information, and/or combination of information.

At step 720, the augmented reality user device 400 presents arecommendation for the user 106 as a virtual object in the real scene infront of the user 106. The augmented reality user device 400 presents arecommendation and any other information provided by the virtualassessment data 111 as virtual objects overlaid on top of the productsor adjacent to the products. When the augmented reality user device 400presents the virtual objects to the user 106, the user 106 can readilysee their current account information.

At step 722, the augmented reality user device 400 determined whetherthere any new accounts available for the user 106. For example, theaugmented reality user device 400 determines that there are new accountsavailable for the user 106 based on the presence of new accounts in thevirtual assessment data 111. In other examples, the augmented realityuser device 400 employs any other technique for determining whetherthere are new accounts available for the user 106. The augmented realityuser device 400 proceeds to step 724 when the augmented reality userdevice 400 determines there are new accounts available for the user 106.The augmented reality user device 400 terminates the method 700 when theaugmented reality user device 400 determines there are no new accountsavailable for the user 106.

At step 724, the augmented reality user device 400 presents theavailable new accounts to the user 106 as virtual objects in the realscene. The augmented reality user device 400 presents the new accountsavailable for the user 106 as virtual objects overlaid on top of theproduct or adjacent to the product. When the augmented reality userdevice 400 presents the virtual objects to the user 106, the user 106can quickly identify any new accounts that are available for the user106 based on their account information, the vendor where they areshopping, and the product. For example, the augmented reality userdevice 400 presents a new credit card account that the user 106 isprequalified for. As another example, the augmented reality user device400 is configured to present a new loan or line of credit that the user106 is prequalified for. As another example, the augmented reality userdevice 400 presents a loyalty or rewards account for a vendor that theuser 106 is qualified to join and is not already a member of. In otherexamples, the augmented reality user device 400 presents any other kindof new accounts the user 106 is qualified for.

At step 726, the augmented reality user device 400 determines whetherthe user 106 accepts any of the new accounts available for the user 106.The augmented reality user device 400 determines that the user 106accepts one of the new accounts when the augmented reality user device400 receives an indication of a selected new account from the user 106as a voice command, a gesture, an interaction with a button on theaugmented reality user device, or in any other suitable form. Theaugmented reality user device 400 may terminate method 700 when theaugmented reality user device 400 determines that the user 106 does notaccept any of the new accounts available for the user 106.

At step 728, the augmented reality user device 400 identifies theselected new account from the user 106. At step 730, the augmentedreality user device 400 sends an activation command 128 identifying theselected new account to the remote server 102.

FIG. 8 is a flowchart of another embodiment of an augmented realityoverlaying method 800 for an augmented reality system 100. In oneembodiment, the augmented reality system 100 is configured similar tothe augmented reality system 100 described in FIG. 1. Method 800 isemployed by the product analysis engine 116 in the server 102 toidentify account information and/or new accounts available for the user106 of an augmented reality user device 400 and provides the requestedinformation to the user 106 in response to receiving a token 110 fromthe augmented reality user device 400.

At step 802, the product analysis engine 116 receives a token 110 froman augmented reality user device 400. The product analysis engine 116decrypts and/or decodes the token 110 when the token 110 is encrypted orencoded by the augmented reality user device 400. In one embodiment, theproduct analysis engine 116 processes the token 110 to identify the useridentifier 108, the product identifier 115, the vendor identifier 117,and the price. In other embodiments, the product analysis engine 116processes the token 110 to identify any other information.

At step 804, the product analysis engine 116 determines accountinformation for the user 106 based on the user identifier 108. Forexample, the product analysis engine 116 uses the user identifier 108 tolook-up the account information for the user 106 in the accountinformation database 120.

At step 806, the product analysis engine 116 identifies one or moreaccounts for the user 106 based on the account information. For example,the product analysis engine 116 queries the account information database120 for existing accounts for the user 106 using the user identifier108.

At step 808, the product analysis engine 116 determines whether thereare new accounts available for the user 106 based on the accountinformation. In one embodiment, the product analysis engine 116 queriesthe account information database 120 for any available new accounts forthe user 106 using the user identifier 108, account information for theuser 106, the product identifiers 115, and/or the vendor identifier 117.In another embodiment, the product analysis engine 116 sends a datarequest 126 to one or more third-party databases 118 to query thethird-party databases 118 for available new accounts for the user 106based on the user identifier 108, the account information for the user106, the product identifiers 115, and/or the vendor identifier 117.

In one embodiment, the product analysis engine 116 prequalifies the user106 for a new account based on the user's 106 account information. Forinstance, the product analysis engine 116 uses a credit history or acredit score for the user 106 to identify new accounts for the user 106,for example, a credit card or a line of credit. As another example, theproduct analysis engine 116 uses the account information to identifyloyalty or reward programs the user 106 is a member of. The productanalysis engine 116 identifies new loyalty or reward program accountsfor the user 106 that the user 106 is not already a member of. In otherexamples, the product analysis engine 116 identifies new accounts forthe user 106 using any other suitable information for the user 106.

At step 810, the product analysis engine 116 selects an account from theone or more accounts and/or the one or more new accounts for the user106 based on the price of the product. In one embodiment, the productanalysis engine 116 selects a particular account for the user 106 basedon the price of the product. For example, when the price of the productis high, the product analysis engine 116 selects an account that allowsfinancing or payments rather than depleting one of the user's 106accounts. In another embodiment, the product analysis engine 116 selectsan account that avoids over drafting or going over budget for the user106. In another embodiment, the product analysis engine 116 selects anaccount that provides some benefit (e.g. reward points) for the user106. In other embodiments, the product analysis engine 116 selects anaccount for the user 106 based on any other criteria. At step 812, theproduct analysis engine 116 generates a recommendation identifying theselected account for the user 106.

At step 814, the product analysis engine 116 proceeds to step 816 whenthe product analysis engine 116 determines there are no new accountsavailable for the user 106. At step 816, the product analysis engine 116generates virtual assessment data 111 comprising the recommendation forthe user 106. The product analysis engine 116 proceeds to step 818 whenthe product analysis engine 116 determines there are new accountsavailable for the user 106. At step 818, the product analysis engine 116generates virtual assessment data 111 comprising the recommendation andidentifying the one or more new accounts available for the user 106. Inone embodiment, the virtual assessment data 111 comprises informationfor the one or more available new accounts. For example, the virtualassessment data 111 comprises interest rate information, terminformation, monthly payment information, or any other information aboutthe new accounts. At step 820, the product analysis engine 116 sends thevirtual assessment data 111 to the augmented reality user device 400.

At step 822, the product analysis engine 116 determines whether theproducts analysis engine 116 has received an activation command 128 fromthe augmented reality user device 400. The product analysis engine 116proceeds to step 824 when the product analysis engine 116 has receivedan activation command. Otherwise, the product analysis engine 116terminates method 800.

At step 824, the product analysis engine 116 activates the new accountselected by the user 106. The received activation command 128 identifiesa selected new account for the user 106. The product analysis engine 116facilitates activating the selected new account. For example, theproduct analysis engine 116 exchanges messages with a third-partydatabase 118 to activate the selected new account. As another example,the product analysis engine 116 updates the offers database 122 toactivate the selected new account. The product analysis engine 116 mayemploy any other suitable technique for activating the selected newaccount.

FIG. 9 is a schematic diagram of another embodiment of an augmentedreality system 100 configured to overlay virtual objects with a realscene. The augmented reality system 100 is configured similar to theaugmented reality system 100 described in FIG. 1. For example, theaugmented reality user device 400, the network 104, the server 102, andthe third-party database 118 are configured similar to augmented realityuser device 400, the network 104, the server 102, and the third-partydatabase 118 described in FIG. 1, respectively.

In FIG. 9, the augmented reality user device 400 is configured toidentify one or more businesses proximate to the user 106. The augmentedreality user device 400 captures an image 407 of buildings, structures,and/or businesses around the user 106. The augmented reality user device400 performs object recognition on the captured image 407 to identifybusinesses proximate to the user 106. The augmented reality user device400 identifies businesses based on structures, branding, signs, logos,and/or any other identifiers associated with the businesses.

The augmented reality user device 400 is configured to generate a token110 that comprises a user identifier 108 for the user 106, a locationidentifier 902 identifying the location of the user 106, user history940 for the user 106, and/or any other information. In one embodiment,the location identifier 902 identifies the location of the user 106using any suitable coordinate system and/or location descriptors.Examples of location identifier 902 include, but are not limited to, alocation name, a physical address, and latitude and longitudecoordinates. In another embodiment, the location identifier 902identifies the location of the user 106 based on buildings, landmarks,and/or businesses proximate to the user 106. For example, the locationidentifier 902 identifies one or more businesses proximate to the user106.

User history 904 comprises user history data or information linked withthe user 106. Examples of user history data include, but are not limitedto, internet search history, transaction history, geographic locationhistory, social media history, shopping lists, wish lists, accountinformation, membership information, biometric information, healthinformation, vitals, and/or any other history linked with the user 106.For example, the user history 904 comprises a history of products theuser 106 recently looked at online but did not purchase. As anotherexample, the user history 904 comprises a history of businesses the user106 recently visited. As another example, the user history 904 comprisestransaction history for identifies businesses proximate to the user 106.As another example, the user history 904 comprises health information,such as vitals history (e.g. heart rate and/or blood sugar history), forthe user 106.

The augmented reality user device 400 is configured to presentgeographic notifications 908 and information from the received virtualassessment data 111 as one or more virtual objects overlaid with thetangible objects 150 in the real scene in front of the user 106. Forexample, the augmented reality user device 400 presents geographicnotifications 908 identifying various offers for the user 106 as virtualobjects overlaid with the objects 150 in front of the user 106.

In FIG. 9, the one or more processors 112 of the server 102 areconfigured to implement a geolocation analysis engine 906. In anembodiment, the geolocation analysis engine 906 is implemented usinglogic units, FPGAs, ASICs, DSPs, or any other suitable hardware.Examples of the geolocation analysis engine 906 in operation aredescribed in detail below and in FIG. 11.

In one embodiment, the geolocation analysis engine 906 is configured toreceive a token 110 and to process the token 110 to identify a useridentifier 108 for the user 106, a location identifier 902 identifyingthe location of the user 106 or businesses proximate to the user 106,user history 904 for the user 106, and/or any other information. In oneembodiment, processing the token 110 comprises decrypting and/ordecoding the token 110 when the token 110 is encrypted or encoded by theaugmented reality user device 400. The geolocation analysis engine 906employs any suitable decryption or decoding technique as would beappreciated by one of ordinary skill in the art.

The geolocation analysis engine 906 is configured to use the useridentifier 108 to look-up and identify account information for the user106 in the account information database 120 in a manner similar to theproduct analysis engine 116 described in FIG. 1. In one embodiment, thegeolocation analysis engine 906 is configured to use the user identifier108 to look-up and identify user history 904 for the user 106. Forexample, the geolocation analysis engine 906 looks up transactionhistory, account information, membership information, and/or any otherhistory for the user 106. The geolocation analysis engine 906 is alsoconfigured to determine whether there are any offers available for theuser 106 based on information provided by the token 110 in a mannersimilar to the product analysis engine 116 described in FIG. 1.

The geolocation analysis engine 906 is configured to generate virtualassessment data 111 that comprises a geolocation notification 908. Ageolocation notification 908 provides personalized information for theuser 106 based on the location of the user 106 and businesses proximateto the user 106. Geolocation notification 908 may comprise informationrelated to special offers, discounts, promotions, places of interest,and/or any other information related to businesses proximate to the user106. Geolocation notifications 908 are presented to the user 106 as oneor more virtual objects overlaid with tangible object in a real scene infront of the user 106. The geolocation analysis engine 906 sends thevirtual assessment data 111 to the augmented reality user device 400.

The memory 114 of the server 102 is operable to store user history 904and geolocation analysis instructions 910. The geolocation analysisinstructions 910 comprise any suitable set of instructions, logic,rules, or code operable to execute the geolocation analysis engine 906.

The following is another non-limiting example of how the augmentedreality system 100 may operate when the user 106 is looking at one ormore businesses. In this example a user 106 is using the augmentedreality user device 400 while walking around. For example, the user 106is walking within a shopping mall or walking along a street. The user106 authenticates themselves before using the augmented reality userdevice 400 by providing credentials (e.g. a log-in and password) or abiometric signal. The augmented reality user device 400 authenticatesthe user 106 based on the user's input and allows the user 106 togenerate and send tokens 110. The augmented reality user device 400identifies the user 106 and a user identifier 108 for the user 106 uponauthenticating the user 106. Once the user 106 has been authenticated,the user identifier 108 is used by other systems and devices (e.g.remote server 102 or a third-party database 118) to identify andauthenticate the user 106 without the user 106 having to provideadditional credentials for each system.

Once the user 106 is authenticated, the augmented reality user device400 identifies the location of the user 106. In one embodiment, theaugmented reality user device 400 identifies the location of the user106 based on the geographic location of the user 106 and/or businessesproximate to the user 106. For example, the augmented reality userdevice 400 uses geographic location information provided by a GPS sensorwith a map database (e.g. a third-party database 118) to determine thelocation of the user 106 and to identify the geographic location of theuser 106.

The user 106 looks at one or more businesses 912 with the augmentedreality user device 400. The augmented reality user device 400 capturesan image 407 of the businesses and performs object recognition and/oroptical character recognition on the captured image 407 to identify thebusinesses. The augmented reality user device 400 identifies businessesbased on their structures, branding, signs, logos, and/or any otherfeatures. For example, the augmented reality user device 400 identifiesa business based on its logo on a sign in front of the business. Inother embodiments, the augmented reality user device 400 identifies thelocation of the user 106 and/or businesses proximate to the user 106using any other suitable technique.

The augmented reality user device 400 generates a token 110 and sendsthe token 110 to the remote server 102. In one embodiment, the augmentedreality user device 400 generates a token 110 comprising the useridentifier 108 for the user 106 and a location identifier 902identifying the location of the user 106 and/or one or more businessesproximate to the user 106. In another embodiment, the token 110 furthercomprises user history 904 for the user 106, any other suitableinformation, and/or combinations of information. The augmented realityuser device 400 encrypts and/or encodes the token 110 prior to sendingthe token 110 to the remote server 102.

The server 102 receives the token 110 and processes the token 110 toidentify the user identifier 108, the location identifier 902, userhistory 904, and/or any other information provided by the token 110. Theserver 102 decrypts or decodes the token 110 when the token 110 isencrypted or encoded by the augmented reality user device 400. Theserver 102 uses the user identifier 108 to look-up account informationand/or accounts for the user 106 in the account information database120. The server 102 is configured to use the user identifier 108 toidentify one or more accounts for the user 106. The server 102 may alsouse the user identifier 108 to look-up user history 904 for the user106. For example, the server 102 identifies transaction history for theuser 106 in the account information database 120 based on the useridentifier 108.

In one embodiment, the server 102 determines whether there are anyoffers available for the user 106 based on the user's accountinformation, the location of the user 106 (e.g. the location identifier902), and/or the user history 904. For example, the server 102identifies discounts or promotions being offered by businessesidentified by the augmented reality user device 400. As another example,the server 102 identifies discounts or promotions available to the user106 based on the location of the user 106 and the user history 904 (e.g.transaction history or internet search history).

In one embodiment, the server 102 queries one or more third-partydatabases 118 for available offers for the user 106 based on the user's106 identity (e.g. the user identifier 108), the location of the user106 (e.g. the location identifiers 902), and/or user history 904. Forinstance, a third-party database 118 linked with the store identified bythe location identifier 904 and provides information related to couponsor promotions offered by the store.

In one embodiment, the server 102 identifies places of interest for theuser 106 based on the location of the user 106 (e.g. the locationidentifier 902) and/or businesses in front of the user 106. For example,the server 102 uses information provided by the location identifier 902to query a third-party database 118 linked with mapping information toidentify places of interest for the user 106. The server 102 identifiesstores, automated teller machines (ATMs), banks, restrooms, and/or anyother places of interest for the user 106.

In one embodiment, the server 102 obtains information related tobusinesses or landmarks proximate to the user 106. For example, thesever 102 sends a data request 126 with information related to thelocation identifier 902 to a third-party database 118 to requestinformation. The server 102 receives information about businesses orlandmarks proximate to the user 106. The information about businesses orlandmarks proximate to the user 106 may include, but is not limited to,hours of operations, schedule, contact information, and servicesoffered.

The server 102 generates one or more geolocation notifications 908 thatcomprises information obtained based on the token 110. The geolocationnotifications 908 may comprise information related to special offers,discounts, promotions, places of interest, and/or any other informationrelated to businesses proximate to the user 106. The server 102generates virtual assessment data 111 that comprises the one or moregeolocation notifications 908. In other examples, the server 102generates virtual assessment data 111 comprising any other type and/orcombination of information. The server 102 then sends the virtualassessment data 111 to the augmented reality user device 400.

The augmented reality user device 400 receives the virtual assessmentdata 111 and processes the virtual assessment data 111 to access theinformation provided by the server 102. In one embodiment, the augmentedreality user device 400 presents the geolocation notifications 908 forthe user 106 as virtual objects 202 overlaid with their correspondingbusinesses 912 in the real scene in front of the user 106 in real time.In other embodiments, the augmented reality user device 400 presentsrecommendations, account information, for the user 106, and/or any otherinformation as virtual objects 202 overlaid with tangible objects in thereal scene.

In FIG. 9, when the user 106 looks at a gym 914, the augmented realitydevice 400 overlays a geolocation notification 916 indicating membershipdiscounts are available at the gym 914. When the user 106 looks at agrocery store 918, the augmented reality user device 400 overlays ageolocation notification 920 indicating a sports drink sale and ageolocation notification 922 indicating a lunch special at a deli in thegrocery store 918. The augmented reality user device 400 also overlays ageolocation notification 924 indicating an ATM is available inside ofthe grocery store 918. When the user 106 looks at an electronics store926, the augmented reality user device 400 overlays a geolocationnotification 928 indicating a product the user 106 previous search for,but did not purchase is available at the electronics store 926. Theaugmented reality user device 400 overlays a geolocation 930 indicatingthe location of the nearest bank as a point of interest for the user106.

FIG. 10 is a flowchart of another embodiment of an augmented realityoverlaying method 1000 for an augmented reality system 100. In oneembodiment, the augmented reality system 100 is configured similar tothe augmented reality system 100 described in FIG. 9. Method 1100 isemployed by the processor 402 of the augmented reality user device 400to generate a token 110 based on a user 106 of the augmented realityuser device 400 and the location of the user 106. The augmented realityuser device 400 uses the token 110 to determine whether there are anyoffers available for the user 106 based on the location of the user 106.The augmented reality user device 400 receives offer and information inresponse to sending the token 110 and presents the received offers andinformation as virtual objects overlaid with tangible object in a realscene in front of the user 106.

At step 1002, the augmented reality user device 400 authenticates a user106. The user 106 authenticates themselves by providing credentials(e.g. a log-in and password) or a biometric signal. The augmentedreality user device 400 authenticates the user 106 based on the user'sinput. The user 106 is able to generate and send tokens 110 using theaugmented reality user device 400 upon authenticating the user 106.

At step 1004, the augmented reality user device 400 identifies a useridentifier 108 for the user 106. Once the user 106 has beenauthenticated, the augmented reality user device 400 identifies the user106 and a user identifier 108 for the user 106. The user identifier 108may be used to identify and authenticate the user 106 in other systems.

At step 1006, the augmented reality user device 400 captures an image407 of a business. In one embodiment, the user 106 provides a command orsignal to the augmented reality user device 400 that triggers the camera406 to capture an image 407 of a business. The image 407 of the businessmay include signs, branding, logos, structures, and/or any suitablefeatures for identifying the business. In another embodiment, theaugmented reality user device 400 and the camera 406 are configured tocontinuously or periodically capture images 407.

At step 1008, the augmented reality user device 400 identifies thebusiness based on the image 407 of the business. For example, theaugmented reality user device 400 identifies the business based on itsstructure and/or based on branding, text, or logos on or around thebusiness. In other examples, the augmented reality user device 400identifies the product based on any other features of the businessand/or using any other suitable technique.

At step 1010, the augmented reality user device 400 determines thelocation of the user 106 based on the identity of the business and thegeographic location of the user 106. In one embodiment, the augmentedreality user device 400 uses geographic location information provided bythe GPS sensor 416 with a map database to determine the location of theuser 106 and to identify the business proximate to the user 106.

At step 1012, the augmented reality user device 400 obtains user historydata for the user 106. For example, user history data comprises locationhistory, interne search history, transaction history, biometric signalhistory, and/or any other kind of history for the user 106. Theaugmented reality user device 400 obtains user history data from amobile device or any other suitable device. In some embodiments, step1012 is optional and may be omitted.

At step 1014, the augmented reality user device 400 generates a token110. In one embodiment, the token 110 comprises the user identifier 108and a location identifier 902. In other embodiments, the token 110comprises user history 904 and/or any other information. At step 1016,the augmented reality user device 400 sends the token 110 to a remoteserver 102.

At step 1018, the augmented reality user device 400 receives virtualassessment data 111 comprising a geolocation notification 908 for aproduct sold or service offered by the business based on the location ofthe user 106. In one embodiment, the virtual assessment data 111comprises a geolocation notification 908 identifying a points ofinterest (e.g. an ATM) and/or any other information.

At step 1020, the augmented reality user device 400 presents thegeolocation notification 908 as a virtual object overlaid within a realscene in front of the user 106. The augmented reality user device 400presents the geolocation notification 908 and any other informationprovided by the virtual assessment data 111 as virtual objects overlaidon top of the business or adjacent to the business. When the augmentedreality user device 400 presents the virtual objects to the user 106,the user 106 can readily see any offers that are available to the user106.

At step 1022, the augmented reality user device 400 determines whetherthe augmented reality user device 400 is configured for continuousoperation. For example, the augmented reality user device 400 determineswhether the augmented reality user device 400 is configured tocontinuously capture images 407 of businesses around the user 106. Theaugmented reality user device 400 returns to step 1006 when theaugmented reality user device 400 is configured for continuousoperation. Otherwise, the augmented reality user device 400 terminatesmethod 1000.

FIG. 11 is a flowchart of another embodiment of an augmented realityoverlaying method 1100 for an augmented reality system 100. In oneembodiment, the augmented reality system 100 is configured similar tothe augmented reality system 100 described in FIG. 9. Method 1100 isemployed by the geolocation analysis engine 906 in the server 102 toidentify available offers for the user 106 of an augmented reality userdevice 400 and provides geolocation notifications 908 to the user 106 inresponse to receiving a token 110 from the augmented reality user device400.

At step 1102, the geolocation analysis engine 906 receives a token 110identifying a user identifier 108 and a location identifier 902. Thegeolocation analysis engine 906 decrypts and/or decodes the token 110when the token 110 is encrypted or encoded by the augmented reality userdevice 400. In one embodiment, the geolocation analysis engine 906processes the token 110 to identify the user identifier 108 and thelocation identifier 902. In other embodiments, the geolocation analysisengine 906 processes the token 110 to identify any other information,for example, user history 904.

At step 1104, the geolocation analysis engine 906 determines accountinformation for the user 106 based on the user identifier 108. Forexample, the geolocation analysis engine 906 uses the user identifier108 to look-up the account information for the user 106 in the accountinformation database 120.

At step 1106, the geolocation analysis engine 906 generates ageolocation notification 908 based on the token 110. In one embodiment,the geolocation analysis engine 906 generates a geolocation notification908 based on user history data for the user 106. The user history datamay be provided by the token 110 or may be determined using the useridentifier 108. In one example, the geolocation analysis engine 906identifies a product or offer based on an internet search history forthe user 106. The geolocation analysis engine 906 determines whether thebusiness sells a product the user 106 recently searched for and/or hasan offer available for the product. When the business sells the productor has an offer available for the product, the geolocation analysisengine 906 generates a geolocation notification 908 identifying theproduct and/or the offer for the product.

In another example, the geolocation analysis engine 906 identifies anunpurchased product based on internet search history and a transactionhistory for the user 106. The geolocation analysis engine 906 determineswhether the business sells a product the user 106 recently searched forand/or has an offer available for the product. The geolocation analysisengine 906 uses the transaction history to determine whether the user106 has already purchased the product. When the business sells theproduct and the user 106 has not already purchased the product, thegeolocation analysis engine 906 generates a geolocation notification 908identifying the unpurchased product and/or an offer for the unpurchasedproduct.

In another example, the geolocation analysis engine 906 identifies aproduct or offer based on a location history for the user 106. Thegeolocation analysis engine 906 determines products or offers the user106 may be interested in based on the recently visited physicallocations. For instance, the geolocation analysis engine 906 determinesthat the user 106 recently visited a gym and may identify water or asale on water for the user 106. The geolocation analysis engine 906generates a geolocation notification 908 identifying the product and/oroffer.

In another example, the geolocation analysis engine 906 identifies aproduct or offer based on the vitals or biometric signals for the user106. For instance, the geolocation analysis engine 906 identifies asports drink or sale on sports drinks when the user's 106 blood sugar islow. The geolocation analysis engine 906 generates a geolocationnotification 908 identifying the product and/or offer.

In another example, the geolocation analysis engine 906 identifies aproduct or offer based on the time of day and transaction history forthe user 106. For instance, the geolocation analysis engine 906determines that it is lunch time and determines the type of food theuser 106 typically purchases at lunch time based on the transactionhistory. The geolocation analysis engine 906 identifies any lunch offersthe user 106 may be interested in. The geolocation analysis engine 906generates a geolocation notification 908 identifying the offers.

In some embodiments, the geolocation analysis engine 906 identifiesplaces of interest for the user 106 based on the location of the user106. For example, the geolocation analysis engine 906 identifies ATMS,banks, restrooms, landmarks, or any other places of interest for theuser. The geolocation analysis engine 906 identifies the places ofinterest for the user 106 based on account information, user historydata, and/or any other information for the user 106. The geolocationanalysis engine 906 generates a geolocation notification 908 identifyingthe places of interest.

At step 1108, the geolocation analysis engine 906 generates virtualassessment data 111 comprising the geolocation notification 908. In someembodiments, the virtual assessment data 111 further comprises accountinformation and/or any other information for the user 106. At step 1110,the geolocation analysis engine 906 sends the virtual assessment data111 to the augmented reality user device 400.

While several embodiments have been provided in the present disclosure,it should be understood that the disclosed systems and methods might beembodied in many other specific forms without departing from the spiritor scope of the present disclosure. The present examples are to beconsidered as illustrative and not restrictive, and the intention is notto be limited to the details given herein. For example, the variouselements or components may be combined or integrated in another systemor certain features may be omitted, or not implemented.

In addition, techniques, systems, subsystems, and methods described andillustrated in the various embodiments as discrete or separate may becombined or integrated with other systems, modules, techniques, ormethods without departing from the scope of the present disclosure.Other items shown or discussed as coupled or directly coupled orcommunicating with each other may be indirectly coupled or communicatingthrough some interface, device, or intermediate component whetherelectrically, mechanically, or otherwise. Other examples of changes,substitutions, and alterations are ascertainable by one skilled in theart and could be made without departing from the spirit and scopedisclosed herein.

To aid the Patent Office, and any readers of any patent issued on thisapplication in interpreting the claims appended hereto, applicants notethat they do not intend any of the appended claims to invoke 35 U.S.C. §112(f) as it exists on the date of filing hereof unless the words “meansfor” or “step for” are explicitly used in the particular claim.

The invention claimed is:
 1. An augmented reality system comprising: anaugmented reality user device for a user comprising: a displayconfigured to overlay virtual objects onto tangible objects in a realscene in real-time; a camera configured to capture images; a globalposition system (GPS) sensor configured to provide the geographiclocation of the user; one or more processors operably coupled to thedisplay, the camera, and the GPS sensor, and configured to implement: anobject recognition engine configured to identify a business based on animage of the business; a virtual assessment engine configured to:authenticate a user based on a user input; identify a user identifierfor the user in response to authenticating the user; generate a locationidentifier identifying the business; obtain user history for the user;obtain product identifiers identifying a plurality of objects in ashopping cart of the user; generate a token comprising:  the userhistory for the user;  the user identifier for the user;  the productidentifiers; and  the location identifier; send the token to a remoteserver; receive virtual assessment data comprising a geolocationnotification identifying:  an offer provided by the business in responseto sending the token, wherein the offer is based at least on the productidentifiers from the token;  a recommended one of a plurality of useraccounts to use for a purchase transaction; and a virtual overlay engineconfigured to present the geolocation notification as a virtual objectoverlaid within a real scene; and the remote server comprising ageolocation analysis engine for the remote server configured to: receivethe token; determine account information for the user based on the useridentifier for the user; identify the offer provided by the businessbased at least on discounts that can be applied to one or more objectsin the shopping cart of the user and the account information for theuser; generate the geolocation notification identifying the offer;generate the virtual assessment data comprising the geolocationnotification; and send the virtual assessment data to the augmentedreality user device.
 2. The system of claim 1, wherein: the user historycomprises an internet search history for the user; and generating thegeolocation notification further comprises: identifying a product basedon the internet search history for the user; determining the businesssells the identified product; and generating the geolocationnotification identifying the product.
 3. The system of claim 1, wherein:the account information comprises a transaction history for the user;the user history comprises an internet search history for the user; andgenerating the geolocation notification further comprises: identifyingan unpurchased product based on the internet search history and thetransaction history for the user; determining the business sells theunpurchased product; and generating the geolocation notificationidentifying the unpurchased product.
 4. The system of claim 1, wherein:the user history comprises a location history identifying recentlyvisited physical locations for the user; and generating the geolocationnotification further comprises: identifying a product based on thelocation history for the user; determining the business sells theidentified product; and generating the geolocation notificationidentifying the product.
 5. The system of claim 1, wherein: the userhistory comprises biometric signals for the user; and generating thegeolocation notification further comprises: identifying a product basedon the biometric signals for the user; determining the business sellsthe product; and generating the geolocation notification identifying theproduct.
 6. The system of claim 1, wherein: the account informationcomprises a transaction history for the user; and generating thegeolocation notification further comprises: identifying a product basedon the time of the day and the transaction history for the user;determining the business sells the product; and generating thegeolocation notification identifying the product.
 7. The system of claim1, wherein the location identifier indicates the geographic location ofthe user.
 8. An augmented reality overlaying method comprising:authenticating, by a virtual assessment engine, a user based on a userinput; identifying, by the virtual assessment engine, a user identifierfor the user in response to authenticating the user; generating, by thevirtual assessment engine, a location identifier identifying thebusiness; obtaining, by the virtual assessment engine, user history forthe user; obtaining, by the virtual assessment engine, productidentifiers identifying a plurality of objects in a shopping cart of theuser; generating, by the virtual assessment engine, a token comprisingthe user history for the user, the user identifier for the user, theproduct identifiers, and the location identifier; sending, by thevirtual assessment engine, the token to a remote server; determining, bya geolocation analysis engine of the remote server, account informationfor the user based on the user identifier; identifying, by thegeolocation analysis engine, (a) the offer provided by the businessbased at least on discounts that can be applied to one or more objectsin the shopping cart of the user and the account information for theuser; and (b) a recommended one of a plurality of user accounts to usefor a purchase transaction; generating, by the geolocation analysisengine, a geolocation notification identifying the offer; generating, bythe geolocation analysis engine, the virtual assessment data comprisingthe geolocation notification; sending, by the geolocation analysisengine, the virtual assessment data to the augmented reality userdevice; and presenting, by a virtual overlay engine, the geolocationnotification as a virtual object overlaid within a real scene.
 9. Themethod of claim 8, wherein the location identifier indicates thegeographic location of the user.
 10. The method of claim 8, wherein: theuser history comprises an internet search history for the user; andgenerating the geolocation notification further comprises: identifying aproduct based on the internet search history for the user; determiningthe business sells the identified product; and generating thegeolocation notification identifying the product.
 11. The method ofclaim 8, wherein: the account information comprises a transactionhistory for the user; the user history comprises an internet searchhistory for the user; and generating the geolocation notificationfurther comprises: identifying an unpurchased product based on theinternet search history and the transaction history for the user;determining the business sells the unpurchased product; and generatingthe geolocation notification identifying the unpurchased product. 12.The method of claim 8, wherein: the user history comprises a locationhistory identifying recently visited physical locations for the user;and generating the geolocation notification further comprises:identifying a product based on the location history for the user;determining the business sells the identified product; and generatingthe geolocation notification identifying the product.
 13. The method ofclaim 8, wherein: the user history comprises biometric signals for theuser; and generating the geolocation notification further comprises:identifying a product based on the biometric signals for the user;determining the business sells the product; and generating thegeolocation notification identifying the product.
 14. The method ofclaim 8, wherein: the account information comprises a transactionhistory for the user; and generating the geolocation notificationfurther comprises: identifying a product based on the time of the dayand the transaction history for the user; determining the business sellsthe product; and generating the geolocation notification identifying theproduct.
 15. An augmented reality user device for a user comprising: adisplay configured to overlay virtual objects onto tangible objects in areal scene in real-time; a camera configured to capture images; a globalposition system (GPS) sensor configured to provide the geographiclocation of the user; one or more processors operably coupled to thedisplay, the camera, and the GPS sensor, and configured to implement: anobject recognition engine configured to identify a business based on animage of the business; a virtual assessment engine configured to:authenticate a user based on a user input; identify a user identifierfor the user in response to authenticating the user; generate a locationidentifier identifying the business; obtain product identifiersidentifying a plurality of objects in a shopping cart of the user;generate a token comprising: the user identifier; the productidentifiers; and the location identifier; send the token to a remoteserver; receive virtual assessment data comprising a geolocationnotification identifying: an offer provided by the business in responseto sending the token, wherein the offer is based at least on discountsthat can be applied to one or more objects in the shopping cart of theuser; a recommended one of a plurality of user accounts to use for apurchase transaction; and a virtual overlay engine configured to presentthe geolocation notification as a virtual object overlaid within a realscene.
 16. The device of claim 15, wherein the location identifierindicates the geographic location of the user.
 17. The device of claim15, wherein: the token comprises user history data comprising aninternet search history for the user; and the geolocation notificationidentifies a product sold by the business based on the internet searchhistory for the user.
 18. The device of claim 15, wherein: the tokencomprises user history data comprising a location history identifyingrecently visited physical locations for the user; and the geolocationnotification identifies a product sold by the business based on thelocation history.
 19. The device of claim 15, wherein: the tokencomprises user history data comprising biometric signals for the user;and the geolocation notification identifies a product sold by thebusiness based on the biometric signals for the user.
 20. The device ofclaim 15, wherein the geolocation notification identifies one or moreplaces of interest.